-- Logs begin at Sat 2022-09-17 03:48:15 UTC, end at Sat 2022-09-17 04:58:21 UTC. -- Sep 17 04:19:54.109260 dv-cinder-857993-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 17 04:19:54.118564 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 04:19:54 2022] *** Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: nodename: dv-cinder-857993-2-iscsi Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: machine: x86_64 Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: clock source: unix Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: pcre jit disabled Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: detected number of CPU cores: 4 Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: current working directory: / Sep 17 04:19:54.120542 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: detected binary path: /usr/bin/uwsgi-core Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: your processes number limit is 47680 Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: your memory page size is 4096 bytes Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: detected max file descriptor number: 2048 Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: lock engine: pthread robust mutexes Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: thunder lock: enabled Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 17 04:19:54.121291 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 04:19:54.172452 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: Python main interpreter initialized at 0x5565a4a6d9c0 Sep 17 04:19:54.172452 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: python threads support enabled Sep 17 04:19:54.172452 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: your server socket listen backlog is limited to 100 connections Sep 17 04:19:54.172452 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: your mercy for graceful operations on workers is 90 seconds Sep 17 04:19:54.172901 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 04:19:54.172901 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: *** Operational MODE: preforking *** Sep 17 04:19:54.173283 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: *** uWSGI is running in multiple interpreter mode *** Sep 17 04:19:54.173283 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: spawned uWSGI master process (pid: 105443) Sep 17 04:19:54.173436 dv-cinder-857993-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 17 04:19:54.177115 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: spawned uWSGI worker 1 (pid: 105444, cores: 1) Sep 17 04:19:54.177115 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: spawned uWSGI worker 2 (pid: 105445, cores: 1) Sep 17 04:19:54.177115 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 04:19:54.818424 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 17 04:19:54.821905 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 17 04:19:55.875437 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:19:55.875437 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: from cryptography.utils import int_from_bytes Sep 17 04:19:55.876079 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:19:55.876079 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: from cryptography.utils import int_from_bytes Sep 17 04:19:55.909273 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:19:55.909273 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: from cryptography.utils import int_from_bytes Sep 17 04:19:55.909910 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:19:55.909910 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: from cryptography.utils import int_from_bytes Sep 17 04:19:58.344312 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:19:58.344907 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:19:58.345058 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:19:58.345450 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:19:58.345700 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:19:58.346081 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.346404 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.346712 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.347036 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.347355 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.347659 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.347969 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.348280 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.348598 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.348834 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.349165 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.349483 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-857993-2-iscsi {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.349793 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.350094 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.350392 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.350701 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.350983 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.351296 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.351594 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.352087 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.352688 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.352992 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.353318 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.353650 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.353956 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.354266 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.354570 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.354870 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.355186 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.355488 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.355887 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-857993-2-iscsi {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.356196 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.356498 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.356696 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:19:58.357170 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.357307 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:19:58.357431 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.357554 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:19:58.357673 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.357803 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.357923 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:19:58.358106 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.358232 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:19:58.358432 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.358558 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.358733 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.358867 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359050 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359175 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359361 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359486 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359671 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359815 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.359981 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360113 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360275 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360400 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360609 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360735 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360855 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.360975 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.361185 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.361313 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.361800 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.361938 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362098 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362224 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-857993-2-iscsi {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362448 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362581 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362719 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362841 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.362960 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363080 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363225 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363365 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363541 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363721 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363859 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.363991 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.364114 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.364236 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.364441 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.364606 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.364742 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365004 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365171 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365342 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365467 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365644 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365780 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.365944 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366068 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366243 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366375 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366542 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366680 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366841 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.366967 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367143 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367268 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367453 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367580 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367759 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.367891 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368109 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368244 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-857993-2-iscsi {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368410 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368541 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368718 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.368856 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369103 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369231 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369351 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369517 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369653 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369835 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.369974 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370148 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370470 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370594 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370848 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.370990 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.371162 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.371462 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.371585 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.371757 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.371991 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.372126 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.372348 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.372550 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.372675 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.372954 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.373118 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.373257 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.373505 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.373630 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.373792 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374042 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374177 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374378 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374521 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374678 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.374977 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375100 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375268 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375476 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375601 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375849 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.375978 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.376146 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.376408 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.376534 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.376726 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.376859 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377027 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377297 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377433 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377628 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377801 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.377926 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.378199 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.378331 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.378508 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.378782 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.378917 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379103 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379236 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379400 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379537 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379699 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.379995 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.380119 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.380302 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.380493 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.380620 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.380883 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381021 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381217 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381472 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381599 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381791 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.381956 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.382123 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.382380 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.382512 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.382728 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.382898 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.383024 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.383249 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.383382 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.383651 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.383922 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384052 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384253 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384421 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384554 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384824 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.384947 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.385145 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.385391 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.385526 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.385723 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.385892 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386017 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386294 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386416 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386582 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.386939 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.387196 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.387428 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.387562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.387781 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.387918 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.388076 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.388293 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.388420 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.388699 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.388902 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.389028 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.389327 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.389454 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.389665 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.389795 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.389992 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.390294 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.390420 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.390611 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.390797 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.390929 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.391208 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.391346 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.391525 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.391742 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.391867 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.392119 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.392254 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.392420 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.392777 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.392911 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.393032 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.393255 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.393390 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.393659 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.393782 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.393969 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.394109 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.394285 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.394550 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.394675 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.394885 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.395008 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.395204 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.395503 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.395636 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.395810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.395983 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.396146 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.396469 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.396607 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.396774 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.396959 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.397119 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.397416 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.397556 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.397722 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.397925 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.398058 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.398314 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.398447 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.398647 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.398871 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.399002 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.399285 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.399409 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.399595 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.399797 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.399930 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.400203 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.400325 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.400510 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.400739 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.400876 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.401123 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.401252 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.401434 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.401715 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.401853 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.402048 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.402180 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.402353 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.402666 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.402792 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.403003 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.403160 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.403293 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.403555 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.403679 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.403861 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.404035 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.404168 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.404343 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.404455 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.404661 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.404796 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.404994 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.405144 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.405326 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.405447 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.405627 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.405754 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.405923 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.406111 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.406235 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.406412 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.406545 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.406825 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.406958 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.407154 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.407370 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.407495 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.407738 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.407862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.408039 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.408311 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.408436 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.408621 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.408857 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.408982 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.409235 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.409347 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.409531 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.409657 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.409847 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.409973 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.410150 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.410263 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.410450 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.410576 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.410767 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.410907 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.411064 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.411259 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.411384 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.411656 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.411778 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.411957 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.412188 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.412312 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.412553 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.412688 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.412853 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.413167 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.413304 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.413481 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.413679 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.413803 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.414010 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.414260 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.414440 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.414583 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.414786 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.414910 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.415025 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.415153 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.415468 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.415777 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.415969 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.416102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.416357 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:19:58.416483 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.416671 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.416793 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.416969 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417112 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417294 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417422 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417595 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417721 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.417891 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418017 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418188 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418404 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418515 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418730 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.418863 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419042 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419166 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419354 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419542 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419674 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.419934 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420067 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420228 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420514 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420639 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420845 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.420987 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.421155 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.421485 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.421618 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.421813 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422044 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422183 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422428 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422551 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422734 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.422859 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423039 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423187 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423390 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423518 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423639 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423794 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.423919 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424112 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424272 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424440 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424566 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424773 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.424899 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425109 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425237 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425429 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425556 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425772 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.425897 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426074 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426200 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426391 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426516 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426688 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426824 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.426994 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427119 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427475 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427641 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427775 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.427954 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428095 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428266 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428398 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428572 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428707 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.428878 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429003 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429199 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429326 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429506 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429641 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.429936 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430112 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430241 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430498 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430634 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430804 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.430939 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431116 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431241 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431416 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431543 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431730 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.431855 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432034 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432159 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432348 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432473 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432642 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432774 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.432947 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433073 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433289 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433492 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433627 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433755 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.433900 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434026 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434203 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434362 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434492 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434613 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434792 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.434927 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435092 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435217 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435394 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435520 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435693 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435826 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.435988 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436120 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436289 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436414 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436595 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436734 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.436894 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437029 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437218 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437344 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437517 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437654 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437825 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.437951 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438125 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438251 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438420 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438547 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438716 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.438842 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439020 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439146 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439332 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439456 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439635 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439758 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.439947 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440167 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440304 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440434 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440556 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440673 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440825 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.440960 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441151 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441279 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441449 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441577 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441748 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.441882 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442044 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442169 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442341 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442467 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442647 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442781 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.442967 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443095 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443261 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443395 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443598 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443816 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.443942 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444062 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444182 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444302 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444463 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444599 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444751 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.444886 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445052 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445191 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445387 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445513 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445696 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.445828 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446037 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446163 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446346 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446474 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446648 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446779 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.446988 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447122 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447379 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447519 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447643 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447763 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.447891 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448012 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448177 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448302 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448480 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448612 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448779 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.448918 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449140 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449280 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449405 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449526 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449711 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.449973 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450100 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450301 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450433 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450610 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450735 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.450924 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451057 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451261 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451384 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451572 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451706 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.451897 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452030 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452217 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452339 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452522 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452645 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452821 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.452957 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453146 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453273 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453449 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453585 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453785 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.453919 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454043 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454172 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454322 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454505 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454640 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454775 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.454897 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455026 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455195 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455321 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455493 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455619 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455791 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.455915 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456087 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456213 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456386 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456512 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456695 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456820 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.456991 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457141 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457326 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457453 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457624 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457750 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.457940 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458073 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458243 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458368 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458541 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458676 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458834 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.458969 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459132 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459258 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459470 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459595 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459772 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.459907 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460103 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460230 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460427 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460553 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460750 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.460877 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461050 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461219 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461375 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461499 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461765 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.461890 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462065 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462188 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462366 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462493 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462664 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462801 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.462969 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463103 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463272 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463408 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463569 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463698 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.463902 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464027 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464155 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464278 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464454 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464620 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464743 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.464864 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465037 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465194 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465365 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465501 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465661 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465787 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.465968 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466108 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466270 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466397 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466564 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466690 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466860 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.466996 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467161 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467287 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467467 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467593 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467761 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.467895 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468063 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468188 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468359 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468484 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468668 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468793 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.468961 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469115 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469293 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469418 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469631 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469759 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.469887 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470010 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470163 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470288 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470459 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470586 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470773 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.470906 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471081 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471207 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471378 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471506 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471673 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471800 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.471974 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472107 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472266 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472382 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472686 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.472861 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473180 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473305 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473491 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473617 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473787 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.473935 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474121 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474244 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474373 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474507 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474669 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474802 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.474927 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475048 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475212 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475337 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475503 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475631 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475796 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.475920 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476086 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476211 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476379 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476505 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476674 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476807 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.476986 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477135 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477350 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477477 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477652 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477786 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.477947 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478073 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478240 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478366 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478574 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478699 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.478869 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479007 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479219 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479456 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479580 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479796 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.479920 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480104 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480238 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480409 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480534 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480713 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.480839 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481019 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481223 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481349 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481469 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481636 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481760 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.481939 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482074 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482242 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482376 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482538 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482665 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482832 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.482970 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483144 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483269 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483577 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483757 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.483891 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484060 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484260 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484396 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484526 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484657 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484826 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.484948 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485068 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485255 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485382 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485558 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485687 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485858 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.485980 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.486159 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.486281 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.486804 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.486934 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487113 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487237 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487416 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487719 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.487851 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488024 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488155 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488325 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488456 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488621 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488744 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.488921 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489046 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489301 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489433 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489555 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489675 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489825 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.489949 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490125 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490256 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490418 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490542 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490719 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.490844 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491013 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491147 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491316 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491441 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491614 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.491739 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492006 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492130 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492310 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492434 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492619 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492744 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.492969 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493117 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493303 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493427 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493608 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493731 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.493917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494040 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494223 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494422 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494547 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494675 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494845 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.494969 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495090 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495210 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495394 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495601 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495727 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.495847 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496003 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496130 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496307 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496440 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496613 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496738 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.496923 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497048 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497262 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497401 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497686 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497861 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.497988 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498160 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498299 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498457 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498581 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498764 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.498887 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499066 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499192 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499370 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499495 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499678 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499802 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.499985 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500114 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500289 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500416 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500613 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500737 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.500931 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501068 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501265 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501404 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501587 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501711 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501866 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.501992 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502170 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502305 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502473 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502607 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502780 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.502906 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503101 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503253 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503408 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503550 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503708 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.503832 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504002 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504125 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504306 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504453 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504641 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504766 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.504931 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505054 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505234 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505379 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505512 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505642 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505787 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.505913 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506087 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506213 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506391 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506516 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506691 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.506817 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507012 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507138 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507323 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507629 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507763 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.507932 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508058 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508234 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508362 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508589 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508715 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.508890 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509015 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509217 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509343 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509525 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509664 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509827 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.509952 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510125 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510257 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510420 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510556 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510737 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.510862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511034 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511159 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511349 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511482 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511656 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511788 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.511964 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512096 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512405 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512568 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512694 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512870 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.512993 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513192 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513318 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513490 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513614 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513785 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.513917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514081 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514218 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514413 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514533 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514741 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.514875 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515041 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515164 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515314 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515437 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515625 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515766 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.515924 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516049 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516230 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516354 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516543 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516675 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516850 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.516978 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.517180 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.517305 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.517478 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.517604 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.517917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518063 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518063 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518306 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518436 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518558 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518680 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518814 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.518947 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519070 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519212 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519341 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519507 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519649 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519807 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.519931 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520101 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520225 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520394 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520533 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520686 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520817 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.520998 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521163 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521320 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521454 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521632 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521766 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.521932 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522057 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522228 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522354 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522523 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522655 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522818 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.522943 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523153 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523279 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523452 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523589 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523776 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.523899 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524154 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524282 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524473 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524606 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524826 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.524966 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525141 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525267 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525421 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525547 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525732 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.525869 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526044 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526169 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526354 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526487 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526662 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526796 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.526963 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527103 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527274 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527401 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527581 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527719 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.527888 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528093 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528218 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528397 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528523 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528719 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.528898 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529024 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529234 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529360 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529540 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529667 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.529923 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530046 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530230 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530356 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530534 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530666 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530834 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.530959 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531134 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531267 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531437 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531565 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531756 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.531881 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532070 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532202 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532380 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532504 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532702 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.532826 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533008 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533166 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533338 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533466 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533644 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533771 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.533947 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534117 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534266 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534391 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534590 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534724 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.534885 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535011 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535192 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535329 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535500 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535626 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535804 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.535938 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536110 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536235 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536412 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536538 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536718 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.536853 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537020 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537196 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537345 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537484 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537651 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537778 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.537956 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538099 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538285 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538419 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538687 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.538888 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539014 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539200 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539334 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539505 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539629 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539838 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.539962 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540141 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540315 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540439 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540653 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540778 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.540964 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541195 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541322 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541444 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541623 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541759 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.541941 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542082 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542252 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542377 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542695 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.542881 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543014 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543179 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543303 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543478 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543605 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543779 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.543904 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544076 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544209 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544374 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544499 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544690 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544816 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.544992 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545163 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545334 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545458 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545615 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545742 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.545918 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546044 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546284 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546440 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546616 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546783 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.546936 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547093 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547252 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547411 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547569 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547733 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.547896 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548030 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548202 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548530 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548659 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548782 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.548903 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549035 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549181 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549318 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549444 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549571 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549693 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549822 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.549982 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550117 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550282 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550409 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550583 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550708 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.550884 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551010 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551206 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551332 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551506 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551631 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551802 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.551930 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552115 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552239 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552411 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552537 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552709 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.552834 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553003 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553146 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553333 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553458 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553632 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553760 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.553928 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554054 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554228 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554371 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554525 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554652 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554829 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.554954 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555123 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555259 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555501 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555629 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555800 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.555925 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556109 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556243 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556423 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556555 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556738 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.556862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557035 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557224 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557375 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557501 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557680 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557807 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.557979 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558106 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558293 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558503 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558640 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558833 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.558948 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559244 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559456 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559561 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559766 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.559874 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560080 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560204 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560393 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560607 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560704 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.560935 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561033 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561277 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561374 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561601 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561703 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.561929 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562028 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562222 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562308 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562522 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562619 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562820 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.562908 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563135 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563238 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563446 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563554 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563736 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.563825 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564035 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564123 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564336 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564425 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564636 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564724 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.564935 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565022 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565259 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565350 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565562 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565657 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565859 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.565961 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566160 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566250 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566461 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566555 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566760 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.566856 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567058 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567146 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567359 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567448 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567661 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567757 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.567972 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568061 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568285 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568373 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568591 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568693 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568897 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.568990 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569226 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569318 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569528 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569617 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569828 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.569928 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570127 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570214 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570426 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570652 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570740 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.570990 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571077 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571323 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571426 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571647 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571741 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.571969 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.572285 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.572500 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.572647 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.572785 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.572921 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573058 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573217 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573380 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573519 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573655 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573814 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.573973 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574110 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574254 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574392 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574529 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574716 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.574860 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575009 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575147 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575315 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575505 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575657 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575808 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.575958 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.576095 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.576231 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.576446 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.576590 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.576794 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577013 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577184 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577334 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577471 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577607 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577744 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.577911 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578050 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578187 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578323 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578458 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578609 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578831 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.578975 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579111 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579247 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579390 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579536 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579674 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579835 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.579972 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580108 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580244 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580411 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580555 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580695 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580839 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.580976 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581130 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581285 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581431 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581578 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581723 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.581861 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582011 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582158 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582295 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582430 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582574 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582711 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582856 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.582994 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583136 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583286 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583439 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583587 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583733 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.583869 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584011 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584155 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584291 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584433 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584582 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584718 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584853 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.584990 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585162 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585328 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585467 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585640 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585781 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.585918 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586055 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586199 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586335 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586478 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586670 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586811 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.586946 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587092 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587240 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587384 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587529 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587667 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587826 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.587980 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588117 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588254 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588426 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588572 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588728 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.588948 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589133 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589279 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589416 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589552 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589706 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589844 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.589980 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590115 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590251 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590518 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590671 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.590946 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591100 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591238 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591408 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591561 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591710 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.591848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592017 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592162 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592446 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592614 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592758 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.592905 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593041 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593244 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593394 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593531 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593677 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593813 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.593958 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594095 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594231 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594448 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594600 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594737 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.594906 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595047 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595191 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595472 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595610 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595802 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.595940 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596084 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596221 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596365 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596501 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596658 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596806 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.596943 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597115 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597271 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597441 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597586 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597753 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.597897 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.598060 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.598204 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.598354 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.598492 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.598640 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:19:58.598786 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.599050 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.599354 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.599667 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.599976 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.600285 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.600608 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.600935 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.601271 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.601577 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.602270 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.602594 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.602920 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.603233 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.603547 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.603856 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.604181 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.604498 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.604821 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.605152 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.605486 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.605826 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.606154 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.606479 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.606787 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.607102 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.607407 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.607717 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.608036 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.608345 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.608649 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.608954 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.609294 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.609597 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.609902 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.610206 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.610508 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.610849 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.611148 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.611801 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.612112 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.612416 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.612720 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.613025 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.613423 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.613727 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.614018 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.614321 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.614632 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.614925 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.615254 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.615588 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.615901 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.616224 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.616539 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.616859 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.617197 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.617521 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.617943 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.618263 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.618575 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.618909 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.619219 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.619532 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.619846 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.620158 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.620468 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.620777 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.621129 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.621461 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.621768 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.622094 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.622418 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.622738 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.623054 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.623367 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.623694 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.624008 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.624319 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.624627 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.624938 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.625278 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.625598 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.625948 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.626259 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.626576 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.626893 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.627230 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.627544 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.627834 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.628156 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.628484 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.628850 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.629201 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.629554 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.629884 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.630200 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.630514 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.630819 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:19:58.631064 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:19:58.753855 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO dbcounter [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Registered counter for database nova_api Sep 17 04:19:58.775740 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_db.sqlalchemy.engines [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.782739 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writer thread running {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:58.791662 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO dbcounter [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Registered counter for database nova_api Sep 17 04:19:58.811835 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_db.sqlalchemy.engines [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.818969 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writer thread running {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:58.838642 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.context [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fa9fccbc-46c5-4ec3-934d-edb50649c947(cell1) {{(pid=105444) load_cells /opt/stack/nova/nova/context.py:464}} Sep 17 04:19:58.842567 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:19:58.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:19:58.844675 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:19:58.845346 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:19:58.845789 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:19:58.847751 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:19:58.849762 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO dbcounter [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Registered counter for database nova_cell0 Sep 17 04:19:58.853893 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO dbcounter [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Registered counter for database nova_cell1 Sep 17 04:19:58.862127 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_db.sqlalchemy.engines [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.863930 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_db.sqlalchemy.engines [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.869460 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writer thread running {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:58.872048 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writer thread running {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:58.876408 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.context [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fa9fccbc-46c5-4ec3-934d-edb50649c947(cell1) {{(pid=105445) load_cells /opt/stack/nova/nova/context.py:464}} Sep 17 04:19:58.880304 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:19:58.880796 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:19:58.882432 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:19:58.883155 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:19:58.883635 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:19:58.885695 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_concurrency.lockutils [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105445) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:19:58.887686 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO dbcounter [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Registered counter for database nova_cell0 Sep 17 04:19:58.891746 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO dbcounter [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Registered counter for database nova_cell1 Sep 17 04:19:58.899211 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_db.sqlalchemy.engines [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.902589 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_db.sqlalchemy.engines [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:58.905538 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writer thread running {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:59.017098 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO dbcounter [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Registered counter for database nova_cell0 Sep 17 04:19:59.024452 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_db.sqlalchemy.engines [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:59.029342 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writer thread running {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:59.043245 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writer thread running {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:59.053466 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO dbcounter [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Registered counter for database nova_cell0 Sep 17 04:19:59.062619 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG oslo_db.sqlalchemy.engines [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:19:59.067652 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writer thread running {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:19:59.361945 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: WARNING keystonemiddleware.auth_token [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:19:59.369355 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: WARNING keystonemiddleware.auth_token [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:19:59.380371 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: WARNING keystonemiddleware.auth_token [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:19:59.387859 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: WARNING keystonemiddleware.auth_token [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:19:59.708472 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: WARNING keystonemiddleware.auth_token [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:19:59.714679 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: WARNING keystonemiddleware.auth_token [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:19:59.719966 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5565a4a6d9c0 pid: 105444 (default app) Sep 17 04:19:59.722053 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:19:59.723216 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] 10.222.0.60 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001739 Sep 17 04:19:59.723910 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 788 bytes} [Sat Sep 17 04:19:59 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:19:59.727963 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: WARNING keystonemiddleware.auth_token [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:19:59.734094 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: WARNING keystonemiddleware.auth_token [None req-bf23ca00-1bbd-4b47-983d-bdd0902f1ef9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:19:59.740322 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5565a4a6d9c0 pid: 105445 (default app) Sep 17 04:21:21.023983 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:21.025008 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 Sep 17 04:21:21.025426 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 2/2] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:21.042837 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_api:SELECT=3 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.043991 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_cell0:SELECT=3 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.044732 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_cell1:SELECT=3 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.045984 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_cell0:SELECT=3 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:21.897745 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-b4deb418-c113-49dd-8099-e37864373f52 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:21.930542 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-b4deb418-c113-49dd-8099-e37864373f52 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.898149 Sep 17 04:21:21.931294 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 1/3] 10.222.0.60 () {60 vars in 1292 bytes} [Sat Sep 17 04:21:21 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 900 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:24.503997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:24.504997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-3638c889-fed4-4fa1-802b-4f65b8cb5b2e None None] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 Sep 17 04:21:24.509437 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 3/4] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:24 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:24.978311 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-20fd6032-2767-4dbf-a9cb-21c9875f41e4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "rxtx_factor": 1.0, "ram": 256, "disk": 1, "name": "cirros256", "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:25.028170 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-20fd6032-2767-4dbf-a9cb-21c9875f41e4 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.516412 Sep 17 04:21:25.028835 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 2/5] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:24 2022] POST /compute/v2.1/flavors => generated 407 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 17 04:21:25.044388 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_api:SELECT=3 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:25.045261 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_cell0:SELECT=3 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:25.049043 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_cell1:SELECT=3 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:25.049865 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:25.254757 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-8d7c46e2-6fef-4a93-9a2a-0a80d81b968f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:25.321104 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-8d7c46e2-6fef-4a93-9a2a-0a80d81b968f admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.283623 Sep 17 04:21:25.321703 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 4/6] 10.222.0.60 () {64 vars in 1336 bytes} [Sat Sep 17 04:21:25 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:27.913262 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-20fd6032-2767-4dbf-a9cb-21c9875f41e4 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:27.915037 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-20fd6032-2767-4dbf-a9cb-21c9875f41e4 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002259 Sep 17 04:21:27.915761 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 3/7] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:27 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:28.538665 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-0c450918-5234-4003-9935-7dee86fe6a24 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "ds512M", "id": "d1", "os-flavor-access:is_public": true, "swap": 0, "vcpus": 1, "ram": 512}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:28.583507 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-0c450918-5234-4003-9935-7dee86fe6a24 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.660994 Sep 17 04:21:28.584523 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 5/8] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:27 2022] POST /compute/v2.1/flavors => generated 404 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:28.598851 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-167f5508-0600-4251-b1a3-90819f2e54c7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:28.644859 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-167f5508-0600-4251-b1a3-90819f2e54c7 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049589 Sep 17 04:21:28.645540 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 4/9] 10.222.0.60 () {64 vars in 1336 bytes} [Sat Sep 17 04:21:28 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:31.256333 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-0c450918-5234-4003-9935-7dee86fe6a24 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:31.257338 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-0c450918-5234-4003-9935-7dee86fe6a24 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Sep 17 04:21:31.257785 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 6/10] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:31 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:31.686818 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-31e05222-9e58-4c18-bdb7-5d976a58efc0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 1, "id": "d2", "ram": 1024, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds1G", "rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:31.706677 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-31e05222-9e58-4c18-bdb7-5d976a58efc0 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.441786 Sep 17 04:21:31.707545 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 5/11] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:31 2022] POST /compute/v2.1/flavors => generated 404 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 17 04:21:31.722978 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-68ccba33-c3cd-460f-8193-43e38908bc82 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:31.755399 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-68ccba33-c3cd-460f-8193-43e38908bc82 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038467 Sep 17 04:21:31.756233 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 7/12] 10.222.0.60 () {64 vars in 1336 bytes} [Sat Sep 17 04:21:31 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:34.329017 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-31e05222-9e58-4c18-bdb7-5d976a58efc0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:34.330207 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-31e05222-9e58-4c18-bdb7-5d976a58efc0 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001512 Sep 17 04:21:34.330917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 6/13] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:34.769691 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-f8f86f52-7e80-45bf-9cd6-dd88001d5450 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "disk": 10, "rxtx_factor": 1.0, "id": "d3", "name": "ds2G", "vcpus": 2, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:34.798915 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-f8f86f52-7e80-45bf-9cd6-dd88001d5450 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.462000 Sep 17 04:21:34.801408 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 8/14] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:34 2022] POST /compute/v2.1/flavors => generated 404 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 17 04:21:34.812275 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-f971d51f-047c-4a6a-aabe-b4fce3773b62 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:34.842880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-f971d51f-047c-4a6a-aabe-b4fce3773b62 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034843 Sep 17 04:21:34.843628 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 7/15] 10.222.0.60 () {64 vars in 1336 bytes} [Sat Sep 17 04:21:34 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:37.417365 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-f8f86f52-7e80-45bf-9cd6-dd88001d5450 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:37.418441 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-f8f86f52-7e80-45bf-9cd6-dd88001d5450 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001670 Sep 17 04:21:37.418854 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 9/16] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:37 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:37.849255 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "ram": 4096, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G", "vcpus": 4, "id": "d4", "disk": 20, "rxtx_factor": 1.0}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:37.869414 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.444078 Sep 17 04:21:37.869982 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 8/17] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:37 2022] POST /compute/v2.1/flavors => generated 404 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:37.883322 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-718d1ee0-bc4a-4514-aeae-794f3537b31a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:37.910712 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-718d1ee0-bc4a-4514-aeae-794f3537b31a admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031506 Sep 17 04:21:37.911334 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 10/18] 10.222.0.60 () {64 vars in 1336 bytes} [Sat Sep 17 04:21:37 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:40.572165 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:40.573659 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002078 Sep 17 04:21:40.574173 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 9/19] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:40 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:41.024359 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-554ce434-8c8e-4fe2-ac22-a5ae142ae5e3 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:41.051302 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-554ce434-8c8e-4fe2-ac22-a5ae142ae5e3 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.469053 Sep 17 04:21:41.051995 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 11/20] 10.222.0.60 () {60 vars in 1292 bytes} [Sat Sep 17 04:21:40 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:21:43.690238 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:43.691223 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-11762a67-efb1-4cfd-acfe-dc472802b698 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 17 04:21:43.691656 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 10/21] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:44.095461 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-3f7dbb8c-efab-4047-978c-24f9a629818c admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "id": "1", "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 1, "name": "m1.tiny", "ram": 512, "swap": 0}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:44.118222 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-3f7dbb8c-efab-4047-978c-24f9a629818c admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.420338 Sep 17 04:21:44.118771 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 12/22] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:43 2022] POST /compute/v2.1/flavors => generated 402 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:44.130965 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-c457b506-f125-49a1-8c16-c7801bf1b83f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:44.159009 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-c457b506-f125-49a1-8c16-c7801bf1b83f admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031780 Sep 17 04:21:44.159813 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 11/23] 10.222.0.60 () {64 vars in 1333 bytes} [Sat Sep 17 04:21:44 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:46.796574 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-3f7dbb8c-efab-4047-978c-24f9a629818c admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:46.797616 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-3f7dbb8c-efab-4047-978c-24f9a629818c admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Sep 17 04:21:46.798007 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 13/24] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:47.187157 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-61a63f54-0465-4eb8-876e-0ce0f0877744 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 2048, "os-flavor-access:is_public": true, "id": "2", "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "m1.small", "swap": 0}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:47.207610 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-61a63f54-0465-4eb8-876e-0ce0f0877744 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.403106 Sep 17 04:21:47.208169 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 12/25] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:46 2022] POST /compute/v2.1/flavors => generated 405 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:47.220782 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-2a0d783b-ce1f-4c1b-94e8-1f1bfca6eb45 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:47.245305 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-2a0d783b-ce1f-4c1b-94e8-1f1bfca6eb45 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028145 Sep 17 04:21:47.245914 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 14/26] 10.222.0.60 () {64 vars in 1333 bytes} [Sat Sep 17 04:21:47 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:21:49.856055 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-61a63f54-0465-4eb8-876e-0ce0f0877744 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:49.857327 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-61a63f54-0465-4eb8-876e-0ce0f0877744 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001572 Sep 17 04:21:49.858046 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 13/27] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:50.282413 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e6e45ae1-05c2-49c2-8589-3d46362293bc admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 2, "swap": 0, "name": "m1.medium", "id": "3", "disk": 40, "ram": 4096}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:50.303338 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-e6e45ae1-05c2-49c2-8589-3d46362293bc admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.438775 Sep 17 04:21:50.304230 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 15/28] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:49 2022] POST /compute/v2.1/flavors => generated 406 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:50.316445 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-73328335-e568-4e31-8d99-b2c6ec8985fd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:50.343679 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-73328335-e568-4e31-8d99-b2c6ec8985fd admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030970 Sep 17 04:21:50.344322 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 14/29] 10.222.0.60 () {64 vars in 1333 bytes} [Sat Sep 17 04:21:50 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:21:52.970544 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e6e45ae1-05c2-49c2-8589-3d46362293bc admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:52.971482 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-e6e45ae1-05c2-49c2-8589-3d46362293bc admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Sep 17 04:21:52.971882 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 16/30] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:53.394339 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-21815edb-0299-42b2-8797-dd7571bfd851 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "ram": 8192, "swap": 0, "id": "4", "disk": 80, "os-flavor-access:is_public": true, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:53.400830 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:53.420663 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-21815edb-0299-42b2-8797-dd7571bfd851 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.442607 Sep 17 04:21:53.429072 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 15/31] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:52 2022] POST /compute/v2.1/flavors => generated 405 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 17 04:21:53.434008 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-f6fd6921-9b0d-4195-a6d8-3a5e9c83f42d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:53.460579 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-f6fd6921-9b0d-4195-a6d8-3a5e9c83f42d admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030240 Sep 17 04:21:53.461548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 17/32] 10.222.0.60 () {64 vars in 1333 bytes} [Sat Sep 17 04:21:53 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:21:56.074311 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-21815edb-0299-42b2-8797-dd7571bfd851 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:21:56.075324 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-21815edb-0299-42b2-8797-dd7571bfd851 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Sep 17 04:21:56.075731 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 16/33] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:21:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:21:56.507168 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "disk": 160, "vcpus": 8, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "m1.xlarge", "id": "5", "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:56.513964 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:21:56.529477 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.447278 Sep 17 04:21:56.546964 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-d52d2845-c317-411e-9f57-7f9a84167ade admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:21:56.554222 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 18/34] 10.222.0.60 () {64 vars in 1283 bytes} [Sat Sep 17 04:21:56 2022] POST /compute/v2.1/flavors => generated 408 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:21:56.577935 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-d52d2845-c317-411e-9f57-7f9a84167ade admin admin] 10.222.0.60 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038637 Sep 17 04:21:56.578518 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 17/35] 10.222.0.60 () {64 vars in 1333 bytes} [Sat Sep 17 04:21:56 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:22:13.489856 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:13.491329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002367 Sep 17 04:22:13.491987 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 19/36] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:22:13 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:13.506169 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:22:13.911697 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-4572e1a5-9ab1-416c-810d-c95f9731d123 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:13.930165 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-4572e1a5-9ab1-416c-810d-c95f9731d123 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.431471 Sep 17 04:22:13.930687 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 18/37] 10.222.0.60 () {60 vars in 1292 bytes} [Sat Sep 17 04:22:13 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:15.244214 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:15.245650 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-55022632-4128-4e31-af17-c22958dba5bf admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002096 Sep 17 04:22:15.246205 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 20/38] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:15 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:15.805314 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-1511b96b-a84e-4e48-9355-7a0ff926f41b admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:15.810880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.wsgi [None req-1511b96b-a84e-4e48-9355-7a0ff926f41b admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-1511b96b-a84e-4e48-9355-7a0ff926f41b admin admin] Request method failure captured: Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept: application/json Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept-Encoding: gzip, deflate Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Connection: keep-alive Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Content-Length: 0 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Host: 10.222.0.60 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-System-Scope: None Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: User-Agent: python-novaclient Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Auth-Token: gAAAAABjJUt3dZfSrjkD4zj2c6jYLf9OiG1kbc9I5vLEHnlkoWbg_VIy5igxv5cHXF9QKvmNHmuLEze2HZ_c_e5dt6VaoO70lcxztWiARliCY0SVbGvBo1MFxwc5S9CvH253W7tGlhAt0ZMg_E7iWZ_gTwgJ7sq7fmYbDAjQVhkI9DFVd0QBEhk Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Id: None Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Name: None Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Identity-Status: Confirmed Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Is-Admin-Project: True Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Id: default Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Name: Default Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Name: admin Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Role: member,reader,admin Sep 17 04:22:15.814145 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Roles: member,reader,admin Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant: admin Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Name: admin Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User: admin Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Id: default Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Name: Default Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Name: admin Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: method: > Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: exception: Flavor 42 could not be found. Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: action_args: {'id': '42'} Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:15.815725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:15.816669 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.817848 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-1511b96b-a84e-4e48-9355-7a0ff926f41b admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105445) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:15.818917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-1511b96b-a84e-4e48-9355-7a0ff926f41b admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.421494 Sep 17 04:22:15.818917 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 19/39] 10.222.0.60 () {60 vars in 1190 bytes} [Sat Sep 17 04:22:15 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 423 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:15.826220 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-41bf23db-6168-43be-9011-1d1e1b5703e4 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:15.848963 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-41bf23db-6168-43be-9011-1d1e1b5703e4 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027030 Sep 17 04:22:15.849590 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 21/40] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:15.860681 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-d63a0a9d-1a62-457c-b6cc-7adaa5988b5f admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:15.882696 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-d63a0a9d-1a62-457c-b6cc-7adaa5988b5f admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026883 Sep 17 04:22:15.883216 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 20/41] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:15.895116 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-9e245eca-ab9d-495a-82f8-1a770d87c1a5 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:15.902150 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.wsgi [None req-9e245eca-ab9d-495a-82f8-1a770d87c1a5 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-9e245eca-ab9d-495a-82f8-1a770d87c1a5 admin admin] Request method failure captured: Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept: application/json Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept-Encoding: gzip, deflate Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Connection: keep-alive Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Content-Length: 0 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Host: 10.222.0.60 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-System-Scope: None Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: User-Agent: python-novaclient Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Auth-Token: gAAAAABjJUt3dZfSrjkD4zj2c6jYLf9OiG1kbc9I5vLEHnlkoWbg_VIy5igxv5cHXF9QKvmNHmuLEze2HZ_c_e5dt6VaoO70lcxztWiARliCY0SVbGvBo1MFxwc5S9CvH253W7tGlhAt0ZMg_E7iWZ_gTwgJ7sq7fmYbDAjQVhkI9DFVd0QBEhk Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Id: None Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Name: None Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Identity-Status: Confirmed Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Is-Admin-Project: True Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Id: default Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Name: Default Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Name: admin Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Role: member,reader,admin Sep 17 04:22:15.905548 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Roles: member,reader,admin Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant: admin Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Name: admin Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User: admin Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Id: default Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Name: Default Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Name: admin Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: method: > Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: exception: Flavor 42 could not be found. Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: action_args: {'id': '42'} Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:15.907102 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:15.908073 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:15.909307 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-9e245eca-ab9d-495a-82f8-1a770d87c1a5 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105444) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:15.910556 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-9e245eca-ab9d-495a-82f8-1a770d87c1a5 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.017676 Sep 17 04:22:15.910556 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 22/42] 10.222.0.60 () {60 vars in 1190 bytes} [Sat Sep 17 04:22:15 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 17 04:22:17.165262 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-d63a0a9d-1a62-457c-b6cc-7adaa5988b5f admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:17.166284 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-d63a0a9d-1a62-457c-b6cc-7adaa5988b5f admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Sep 17 04:22:17.166676 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 21/43] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:17.712727 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e51a3204-2b6e-4089-8172-802becc52b3d admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:17.717918 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.wsgi [None req-e51a3204-2b6e-4089-8172-802becc52b3d admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e51a3204-2b6e-4089-8172-802becc52b3d admin admin] Request method failure captured: Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept: application/json Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept-Encoding: gzip, deflate Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Connection: keep-alive Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Content-Length: 0 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Host: 10.222.0.60 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-System-Scope: None Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: User-Agent: python-novaclient Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Auth-Token: gAAAAABjJUt5v3xEVXnkm8oX-0V6FxTGdg1jtqo-BHAZUJQ87Wz5dIKs7pdKoIoIk6XVxZ5jEAba-RyDH9klVz3nuGn7NlV5YhvAqWikgsIDNzvQXEkoRx1B4dBUJCdiK4TWpg9MfO4aN0mHU26du3z897ec-a5xmeU25BonNH1sGIAiHmAgLfI Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Id: None Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Name: None Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Identity-Status: Confirmed Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Is-Admin-Project: True Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Id: default Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Name: Default Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Name: admin Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Role: member,reader,admin Sep 17 04:22:17.719429 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Roles: member,reader,admin Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant: admin Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Name: admin Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User: admin Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Id: default Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Name: Default Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Name: admin Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: method: > Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: exception: Flavor 84 could not be found. Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: action_args: {'id': '84'} Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:17.720997 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:17.723273 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.724602 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e51a3204-2b6e-4089-8172-802becc52b3d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105444) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:17.725895 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-e51a3204-2b6e-4089-8172-802becc52b3d admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.417521 Sep 17 04:22:17.725895 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 23/44] 10.222.0.60 () {60 vars in 1190 bytes} [Sat Sep 17 04:22:17 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 419 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:17.731079 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-7cf0119c-609a-436d-99bc-74de40792842 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:17.746880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-7cf0119c-609a-436d-99bc-74de40792842 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019597 Sep 17 04:22:17.747395 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 22/45] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:17 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:17.756806 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-6885b1db-bec4-42a8-ba4c-3db1ee2daef1 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:17.772784 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-6885b1db-bec4-42a8-ba4c-3db1ee2daef1 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019503 Sep 17 04:22:17.773339 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 24/46] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:17 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:17.782762 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-885d32ca-ce44-4016-82e0-1ad88d7d8fad admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:17.787888 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.wsgi [None req-885d32ca-ce44-4016-82e0-1ad88d7d8fad admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-885d32ca-ce44-4016-82e0-1ad88d7d8fad admin admin] Request method failure captured: Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept: application/json Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept-Encoding: gzip, deflate Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Connection: keep-alive Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Content-Length: 0 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Host: 10.222.0.60 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-System-Scope: None Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: User-Agent: python-novaclient Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Auth-Token: gAAAAABjJUt5v3xEVXnkm8oX-0V6FxTGdg1jtqo-BHAZUJQ87Wz5dIKs7pdKoIoIk6XVxZ5jEAba-RyDH9klVz3nuGn7NlV5YhvAqWikgsIDNzvQXEkoRx1B4dBUJCdiK4TWpg9MfO4aN0mHU26du3z897ec-a5xmeU25BonNH1sGIAiHmAgLfI Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Id: None Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Name: None Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Identity-Status: Confirmed Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Is-Admin-Project: True Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Id: default Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Name: Default Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Name: admin Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Role: member,reader,admin Sep 17 04:22:17.789427 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Roles: member,reader,admin Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant: admin Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Name: admin Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User: admin Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Id: default Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Name: Default Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Name: admin Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: method: > Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: exception: Flavor 84 could not be found. Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: action_args: {'id': '84'} Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:17.791000 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:17.792065 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:17.793725 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-885d32ca-ce44-4016-82e0-1ad88d7d8fad admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105445) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:17.795095 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-885d32ca-ce44-4016-82e0-1ad88d7d8fad admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011670 Sep 17 04:22:17.795095 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 23/47] 10.222.0.60 () {60 vars in 1190 bytes} [Sat Sep 17 04:22:17 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:19.047236 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-6885b1db-bec4-42a8-ba4c-3db1ee2daef1 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:19.048238 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-6885b1db-bec4-42a8-ba4c-3db1ee2daef1 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Sep 17 04:22:19.048642 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 25/48] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:19.604498 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-b5b8594d-1d9b-4dda-834b-c8baf215a10b admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:19.610811 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.wsgi [None req-b5b8594d-1d9b-4dda-834b-c8baf215a10b admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-b5b8594d-1d9b-4dda-834b-c8baf215a10b admin admin] Request method failure captured: Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept: application/json Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Accept-Encoding: gzip, deflate Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Connection: keep-alive Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Content-Length: 0 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Host: 10.222.0.60 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Openstack-System-Scope: None Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: User-Agent: python-novaclient Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Auth-Token: gAAAAABjJUt6Qw5zPSwAFNjPD7_3n3KoTZhwrfXO3VKCqohxp_iStGriQNIk8TesbSG7ysUrPzlXGWuN9IthFEJLb8aA6DnnMix9JrzzNaQuFSJ3XbSh2-u8kcnZ6bsOQSmXK6UwrdSYp-Uh4k9-PHSvTpE90qNs8Bdmze0HM35SWqEIKS4iQCU Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Id: None Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Domain-Name: None Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Identity-Status: Confirmed Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Is-Admin-Project: True Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Id: default Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Domain-Name: Default Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Project-Name: admin Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Role: member,reader,admin Sep 17 04:22:19.612317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Roles: member,reader,admin Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant: admin Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-Tenant-Name: admin Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User: admin Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Id: default Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Domain-Name: Default Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: X-User-Name: admin Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: method: > Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: exception: Flavor 451 could not be found. Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: action_args: {'id': '451'} Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:19.613862 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:19.614804 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.615880 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-b5b8594d-1d9b-4dda-834b-c8baf215a10b admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105445) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:19.616868 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-b5b8594d-1d9b-4dda-834b-c8baf215a10b admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.429043 Sep 17 04:22:19.616868 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 24/49] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:22:19 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 430 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:19.624034 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-e0061ea5-1cc9-462c-8cd0-c1e0160e32b4 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:19.639043 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-e0061ea5-1cc9-462c-8cd0-c1e0160e32b4 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018685 Sep 17 04:22:19.639583 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 26/50] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:19 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:22:19.649132 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:19.664627 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018940 Sep 17 04:22:19.665409 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 25/51] 10.222.0.60 () {60 vars in 1210 bytes} [Sat Sep 17 04:22:19 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:22:19.674838 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-2c7e7932-cd73-46e6-a0db-e7ff3618a8b5 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:19.680188 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.wsgi [None req-2c7e7932-cd73-46e6-a0db-e7ff3618a8b5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-2c7e7932-cd73-46e6-a0db-e7ff3618a8b5 admin admin] Request method failure captured: Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept: application/json Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Accept-Encoding: gzip, deflate Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Connection: keep-alive Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Content-Length: 0 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Host: 10.222.0.60 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-Api-Version: compute 2.92 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Openstack-System-Scope: None Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: User-Agent: python-novaclient Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Auth-Token: gAAAAABjJUt6Qw5zPSwAFNjPD7_3n3KoTZhwrfXO3VKCqohxp_iStGriQNIk8TesbSG7ysUrPzlXGWuN9IthFEJLb8aA6DnnMix9JrzzNaQuFSJ3XbSh2-u8kcnZ6bsOQSmXK6UwrdSYp-Uh4k9-PHSvTpE90qNs8Bdmze0HM35SWqEIKS4iQCU Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Id: None Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Domain-Name: None Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Identity-Status: Confirmed Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Is-Admin-Project: True Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Openstack-Nova-Api-Version: 2.92 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Id: default Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Domain-Name: Default Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Project-Name: admin Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Role: member,reader,admin Sep 17 04:22:19.681810 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Roles: member,reader,admin Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/7fd2fb11102f467d836ef30d6efdd922"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant: admin Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Id: 7fd2fb11102f467d836ef30d6efdd922 Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-Tenant-Name: admin Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User: admin Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Id: default Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Domain-Name: Default Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Id: 8a7b708ff6c140a3ba833eb00eff7926 Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: X-User-Name: admin Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: method: > Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: exception: Flavor 451 could not be found. Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: action_args: {'id': '451'} Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:22:19.683329 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:19.684451 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: ERROR nova.api.openstack.wsgi Sep 17 04:22:19.685866 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-2c7e7932-cd73-46e6-a0db-e7ff3618a8b5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105444) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:22:19.687123 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-2c7e7932-cd73-46e6-a0db-e7ff3618a8b5 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012090 Sep 17 04:22:19.687123 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 27/52] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:22:19 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 17 04:22:20.932470 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:20.933521 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001475 Sep 17 04:22:20.933919 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 26/53] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:21.484606 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-45bd6251-db9a-4584-a1d8-cabad581a1c2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:22:21.507502 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-45bd6251-db9a-4584-a1d8-cabad581a1c2 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.438212 Sep 17 04:22:21.508052 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 28/54] 10.222.0.60 () {64 vars in 1235 bytes} [Sat Sep 17 04:22:21 2022] POST /compute/v2.1/flavors => generated 404 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:22:22.729229 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:22.730248 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Sep 17 04:22:22.730618 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 27/55] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:23.277231 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-585ea32e-f852-4960-b71e-dd2fe5e1e7b6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:22:23.298831 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-585ea32e-f852-4960-b71e-dd2fe5e1e7b6 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.431365 Sep 17 04:22:23.299380 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 29/56] 10.222.0.60 () {64 vars in 1235 bytes} [Sat Sep 17 04:22:22 2022] POST /compute/v2.1/flavors => generated 405 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:22:24.535015 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:24.536032 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Sep 17 04:22:24.536432 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 28/57] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:25.084879 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-d756d24e-6671-4dd6-818a-7aada0a52c36 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:22:25.105739 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-d756d24e-6671-4dd6-818a-7aada0a52c36 admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.433363 Sep 17 04:22:25.106279 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 30/58] 10.222.0.60 () {64 vars in 1235 bytes} [Sat Sep 17 04:22:24 2022] POST /compute/v2.1/flavors => generated 407 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:22:26.328885 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:26.329935 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 Sep 17 04:22:26.330317 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 29/59] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:26.883891 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-2f0274de-3093-41b0-a94d-56b0fd2d45dd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:22:26.908380 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-2f0274de-3093-41b0-a94d-56b0fd2d45dd admin admin] 10.222.0.60 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.442667 Sep 17 04:22:26.908914 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 31/60] 10.222.0.60 () {64 vars in 1235 bytes} [Sat Sep 17 04:22:26 2022] POST /compute/v2.1/flavors => generated 409 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:22:28.137137 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:28.138182 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001481 Sep 17 04:22:28.138543 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 30/61] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:22:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:22:28.697052 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-ff6906e6-f295-48e2-b919-dff360b8736a admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:22:28.719351 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-ff6906e6-f295-48e2-b919-dff360b8736a admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.446893 Sep 17 04:22:28.720061 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 32/62] 10.222.0.60 () {60 vars in 1202 bytes} [Sat Sep 17 04:22:28 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:23:26.098360 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.099973 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-fa29eab0-37fb-4b5e-afb8-03ed0cd751af admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002381 Sep 17 04:23:26.100580 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 31/63] 10.222.0.60 () {54 vars in 931 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:23:26.118432 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:26.526441 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-dd222f57-bcb7-446c-9086-f32cec036a99 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.546777 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-dd222f57-bcb7-446c-9086-f32cec036a99 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.435893 Sep 17 04:23:26.547521 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 33/64] 10.222.0.60 () {60 vars in 1292 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:23:26.568005 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_api:SELECT=14 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:23:26.571731 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-93b11603-8d3c-4d66-a9ac-ff1a9852c1c5 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.579273 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-93b11603-8d3c-4d66-a9ac-ff1a9852c1c5 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012702 Sep 17 04:23:26.579781 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 32/65] 10.222.0.60 () {60 vars in 1283 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:23:26.591501 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-8fdd462a-8c54-4cb6-a14c-6c4e03ff9d2a admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.598849 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-8fdd462a-8c54-4cb6-a14c-6c4e03ff9d2a admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011099 Sep 17 04:23:26.599361 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 34/66] 10.222.0.60 () {60 vars in 1286 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:23:26.611045 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.618297 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011153 Sep 17 04:23:26.618801 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 33/67] 10.222.0.60 () {60 vars in 1286 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:23:26.634005 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-09f91ed5-a8ff-4428-af8e-16d9456cccba admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:23:26.641478 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-09f91ed5-a8ff-4428-af8e-16d9456cccba admin admin] 10.222.0.60 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011054 Sep 17 04:23:26.642021 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 35/68] 10.222.0.60 () {60 vars in 1283 bytes} [Sat Sep 17 04:23:26 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:26:23.745868 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:23.746916 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Sep 17 04:26:23.747311 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 34/69] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:26:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:26:23.888640 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_api:SELECT=6 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:24.070236 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:24.075118 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:24.075634 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:24.076059 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:24.076761 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:24.077288 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:24.077699 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:24.109878 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-86d87e11-3e4b-4b2f-a519-37ffa663683b admin admin] 10.222.0.60 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.223178 Sep 17 04:26:24.110400 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 36/70] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:26:23 2022] GET /compute/v2.1/os-services => generated 791 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:26:25.193932 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:25.195252 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002082 Sep 17 04:26:25.195762 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 35/71] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:26:25 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:26:25.397118 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:25.400493 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:25.401007 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:25.401458 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:25.402170 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:25.402622 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:25.403031 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:25.422529 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-5080fcb5-de25-4977-aa55-425755e20cc6 admin admin] 10.222.0.60 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.078965 Sep 17 04:26:25.423135 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 37/72] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:26:25 2022] GET /compute/v2.1/os-services => generated 791 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:26:36.928488 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:36.929573 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001494 Sep 17 04:26:36.929956 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 36/73] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:26:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:26:37.078983 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_cell0:SELECT=4 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:37.080086 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_cell1:SELECT=4 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:37.081176 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG dbcounter [-] [105444] Writing DB stats nova_api:SELECT=8 {{(pid=105444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:26:37.137741 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:37.141117 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:37.141617 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:37.142035 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:37.142742 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:37.143177 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:37.143572 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:37.168235 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-ded376fb-6bad-4abe-ac64-e0362f1e0cee admin admin] 10.222.0.60 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.096845 Sep 17 04:26:37.168808 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 38/74] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:26:37 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:26:38.272149 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG nova.api.openstack.wsgi [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] Calling method '>' {{(pid=105445) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:38.273185 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: INFO nova.api.openstack.requestlog [None req-de498a3a-6f4b-48bd-a4fe-ea0267bad9d0 admin admin] 10.222.0.60 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001435 Sep 17 04:26:38.273572 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: [pid: 105445|app: 0|req: 37/75] 10.222.0.60 () {56 vars in 1074 bytes} [Sat Sep 17 04:26:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:26:38.469419 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG nova.api.openstack.wsgi [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Calling method '>' {{(pid=105444) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:26:38.472900 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:38.473342 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:38.473761 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:38.474458 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:26:38.474898 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:26:38.475306 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: DEBUG oslo_concurrency.lockutils [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105444) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:26:38.497169 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: INFO nova.api.openstack.requestlog [None req-23220f32-0338-4558-b38c-9bff39274eba admin admin] 10.222.0.60 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.087331 Sep 17 04:26:38.497711 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: [pid: 105444|app: 0|req: 39/76] 10.222.0.60 () {60 vars in 1193 bytes} [Sat Sep 17 04:26:38 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:27:35.330421 dv-cinder-857993-2-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 17 04:27:35.331091 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: Sat Sep 17 04:27:35 2022 - graceful shutdown triggered... Sep 17 04:27:35.331547 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: Gracefully killing worker 2 (pid: 105445)... Sep 17 04:27:35.331924 dv-cinder-857993-2-iscsi devstack@n-api.service[105444]: Gracefully killing worker 1 (pid: 105444)... Sep 17 04:27:35.461110 dv-cinder-857993-2-iscsi devstack@n-api.service[105445]: DEBUG dbcounter [-] [105445] Writing DB stats nova_api:SELECT=4 {{(pid=105445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:27:36.332054 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: worker 1 buried after 1 seconds Sep 17 04:27:36.332054 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: worker 2 buried after 1 seconds Sep 17 04:27:36.332054 dv-cinder-857993-2-iscsi devstack@n-api.service[105443]: goodbye to uWSGI. Sep 17 04:27:36.334577 dv-cinder-857993-2-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 17 04:27:36.335547 dv-cinder-857993-2-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 17 04:27:36.339994 dv-cinder-857993-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 17 04:27:36.352681 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 17 04:27:36.355648 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 04:27:36 2022] *** Sep 17 04:27:36.355797 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 04:27:36.355925 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 04:27:36.356042 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: nodename: dv-cinder-857993-2-iscsi Sep 17 04:27:36.356155 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: machine: x86_64 Sep 17 04:27:36.356275 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: clock source: unix Sep 17 04:27:36.356394 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: pcre jit disabled Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: detected number of CPU cores: 4 Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: current working directory: / Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: detected binary path: /usr/bin/uwsgi-core Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: your processes number limit is 47680 Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: your memory page size is 4096 bytes Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: detected max file descriptor number: 2048 Sep 17 04:27:36.356554 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: lock engine: pthread robust mutexes Sep 17 04:27:36.357067 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: thunder lock: enabled Sep 17 04:27:36.357067 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 17 04:27:36.357067 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: Python main interpreter initialized at 0x55c3f529b9c0 Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: python threads support enabled Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: your server socket listen backlog is limited to 100 connections Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: your mercy for graceful operations on workers is 90 seconds Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: *** Operational MODE: preforking *** Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: *** uWSGI is running in multiple interpreter mode *** Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: spawned uWSGI master process (pid: 120614) Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: spawned uWSGI worker 1 (pid: 120626, cores: 1) Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: spawned uWSGI worker 2 (pid: 120627, cores: 1) Sep 17 04:27:36.399780 dv-cinder-857993-2-iscsi devstack@n-api.service[120614]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 04:27:36.396710 dv-cinder-857993-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 17 04:27:37.051493 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 17 04:27:37.080338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 17 04:27:38.176658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:27:38.176658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: from cryptography.utils import int_from_bytes Sep 17 04:27:38.177386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:27:38.177386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: from cryptography.utils import int_from_bytes Sep 17 04:27:38.217413 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:27:38.217413 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: from cryptography.utils import int_from_bytes Sep 17 04:27:38.218014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 04:27:38.218014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: from cryptography.utils import int_from_bytes Sep 17 04:27:40.551383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:27:40.552400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:27:40.552400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:27:40.552604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:27:40.552785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:27:40.553166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.553517 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.553846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.554162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.554494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.554807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.555134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.555452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.555790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.556048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.556364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.556865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-857993-2-iscsi {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.557239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.557543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.558053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.558388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.558697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.559023 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.559321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.559912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.562332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.563852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.563852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.563852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.563852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.564141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-857993-2-iscsi {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.564268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.564575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.564871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.565390 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.565678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.565989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.566313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.566634 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.566945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.567276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.567608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.567912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.568230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.568535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.568842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.569176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.569495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.569801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.570106 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.570350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.570656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.570983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.571302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.571624 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.571875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.572124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.572439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.572758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.573068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.573398 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.573702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.574009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.574314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.574616 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.574920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.575229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.575564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.575862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.576183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.576489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.576806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.577242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.577499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.577805 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.578152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.578549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.578884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.579219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.579545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.579853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.580167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.580472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.580791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.581196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.581469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.581721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.582025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.582330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.582649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.582955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.583259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.583564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.583864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.584170 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.584491 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.584779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.585097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.585419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.585841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.586376 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.586857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.587177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.587489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.587818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.588140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.588451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.588760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.589072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.589417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.589742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.590050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.590361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.590696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.591014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.591321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.591639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.591946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.592253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.592577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.592868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.593202 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.593524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.593869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.594182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.594485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.594789 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.595096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.595401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.595732 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.596008 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.596306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.596628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.596970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.597289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.597631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.597908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.598229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.598571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.598900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.599239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.599556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.599887 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.600218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.600537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.600849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.601194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.601522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.601835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.602144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.602454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.602768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.603084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.603740 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.604059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.604427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.604691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.604998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.605350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.605670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.605998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.606308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.606646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.606988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.607323 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.607643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.607972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.608282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.608770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.608884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.609255 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.609638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.609959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.610272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.610570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.611060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.611590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.612055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.612372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.612669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.612991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.613366 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.613685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.614009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.614326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.614638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.614958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.615279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.615590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.615901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.616207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.616513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.616816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.617149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.617457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.617777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.618108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.618416 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.618752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.619057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.619369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.619681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.619992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.620298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.620608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.620928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.621258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.621579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.621890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.622196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.622519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.622825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.623131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.623446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.623756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.624066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.624377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.624687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.625009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.625380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.625693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.626025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.626357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.626662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.626986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.627314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.627632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.627935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.628244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.628548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.628852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.628953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 04:27:40.629466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 04:27:40.629562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.629562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.629727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 04:27:40.629852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.630022 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 04:27:40.630174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.630322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 04:27:40.630451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.630659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.630763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.631003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.631138 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.631311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.631404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.631630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.631730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.631939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.632040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.632258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.632365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.632546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.632703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.632890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.633017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.633213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.633380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.633551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.633668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.633888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.634013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.634206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.634401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-857993-2-iscsi {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.634528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.634732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.634850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.635047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.635175 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.635403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.635532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.635686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.635811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.635993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.636126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.636312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.636436 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.636617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.636741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638138 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.638408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638504 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.638590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.638853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.638951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.639036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.639118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.639238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.639373 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.639564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.639676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.639913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.640087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.640274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.640386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.640559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.640654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.640914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.641024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.641311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.641439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.641629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.641800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-857993-2-iscsi {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.641949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.642108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.642656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.642764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.642848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.642921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.643001 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.643149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.643318 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.643481 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.643579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.643764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.643902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.644118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.644285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.644392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.644536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.644730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.644873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.645032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.645168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.645364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.645551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.645648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.645750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.646013 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.646173 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.646308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.646420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.646638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.646789 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.646960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.647121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.647224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.647320 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.647571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.647766 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.647907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.648037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.648139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.648223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.648386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.648526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.648746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.648898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.649068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.649242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.649369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.649458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.649624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.649734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.649912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.650059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.650236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.650396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.650528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.650639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.650861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.651013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.651145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.651256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.651483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.651635 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.651765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.651873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.652101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.652258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.652373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.652459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.652704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.652856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.652992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.653140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.653328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.653480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.653649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.653761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.653960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.654056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.654266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.654370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.654572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.654675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.654874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.654978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.655219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.655322 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.655529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.655632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.655830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.655933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.656146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.656241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.656504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.656600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.656820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.656932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.657168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.657268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.657484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.657597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.657804 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.657951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.658110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.658251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.658430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.658652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.658778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.658898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.659016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.659133 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.659305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.659439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.659610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.659815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.659941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.660060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.660225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.660356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.660534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.660655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.660842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.660969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.661189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.661308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.661496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.661623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.661805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.661922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.662112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.662249 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.662425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.662750 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.662892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.663018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.663163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.663321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.663453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.663640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.663851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.663993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.664127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.664283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.664414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.664622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.664744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.664944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.665293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.665594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.665902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.666633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.667060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.667367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.667676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.667799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.667988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.668117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.668301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.668430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.668741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.668885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.669005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.669169 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.669295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.669416 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.669536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.669654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.669942 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.670265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.670590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.670897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.671219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.671522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.671850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.672288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.672763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.677139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.677139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.677906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.677906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.678205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.678374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.678532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678604 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.678688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.678846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.678924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679677 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.679851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.679927 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680085 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.680993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.680993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.681234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.681392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.681541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.681755 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.681833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 04:27:40.682073 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.682189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.682514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.682820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683393 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.683853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684449 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684601 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.684922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.685956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.686162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.686328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.686485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.686638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687323 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.687965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688409 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.688862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689720 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.689874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690164 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.690903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691202 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.691814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692553 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.692971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.693899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694346 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694491 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.694934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.695866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696023 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.696629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.697351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.697508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.697668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.697814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.697967 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.698954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.699721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.700919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701573 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.701908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702269 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.702914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.703963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704413 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.704867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705644 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705809 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.705963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.706843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.707471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708783 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.708934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.709878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.710873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.711900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712501 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712655 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.712998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.713293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.713443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.713588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.713740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.713894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714039 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.714849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.715900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.716835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.717144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.717698 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.717976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.718990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719573 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.719886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720039 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.720922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.721889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722624 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.722920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.723997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.724909 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725829 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.725982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.726886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.727030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.727176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.727348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.727885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.728047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.728209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.728363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.728832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.728984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729465 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.729905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.730060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.730376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.731863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732158 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732589 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.732888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733740 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.733906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734204 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.734922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735073 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735371 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.735962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.736995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.737156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.737317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.737487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.738928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739091 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.739974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.740126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.740280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.740571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.740732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.740878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.741970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742576 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742719 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.742864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.743830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.744877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745035 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.745975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746119 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.746924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.747084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.747230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.747381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.747537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.747697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.748236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.748394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.748556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.748711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.748940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.749128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.750437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.750721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.750888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.751972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752409 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.752859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.753989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754287 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.754899 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755766 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.755925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756086 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.756867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757228 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.757823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.758437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.758593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.758738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.758902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759212 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.759963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.760728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.761883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762172 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.762938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763092 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.763904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.764256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.764431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.764581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.764725 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.764877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.765967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.766258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.766258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.766551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.766695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.766839 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767316 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767461 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.767919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.768099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.768690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.768845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.768999 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.769215 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.769364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.769539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.770313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.770475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.770622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.770776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.770942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.771778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.772914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.773903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774062 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774396 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.774881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775493 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.775963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.776121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.776291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.776551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.776724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.776877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.777873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.778031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.778189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.778189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.778854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.779971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.780925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.781114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.781424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.781424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.781739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.781890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.782864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.783853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784493 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.784961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785493 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.785970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.786903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787220 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787220 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.787864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.788023 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.788182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.788333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.788333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.788650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.789167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.789327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.789488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.789640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.789835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.790905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.791059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.794146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.794330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.794495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.794675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.794846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795644 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.795954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.796844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797033 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.797862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798022 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.798846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.799488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.799652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.799652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.799949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.800749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.800749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.801891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.802859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.803892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.804974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805175 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805717 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.805891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806209 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806290 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806804 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.806957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807114 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807114 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.807936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.808934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.809927 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.810928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.811856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812204 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812789 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.812977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813065 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813065 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813378 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813752 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813839 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.813998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814713 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.814961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:27:40.815553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.815984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.816461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.817022 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.818170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.818170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.818170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.818644 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.818794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.819086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.819579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.819757 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.820031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.820362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.820900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.821266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.821648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.821916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.822277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.822594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.822924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.823261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.823599 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.823916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.824255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.824591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.824918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.825298 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.825628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.825961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.826302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.826612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.826930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.827243 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.827557 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.827869 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.828184 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.828544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.828854 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.829972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.829972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.829972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.830464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.830464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.830889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.831205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.831518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.831893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.832201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.832541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.832861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.833215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.833544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.833866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.834189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.834510 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.834831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.835153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.835474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.836463 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.836802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.837170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.837509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.838093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.838438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.845150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.847777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.848722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.849753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.850628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.856500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.856940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.857275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.857653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.857983 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.858586 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.858932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.859293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.859706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.860051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.860392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.860732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.861123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.862033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.862033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.862929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.863296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.863873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.864227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.864566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.864888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.865246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.865576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.866147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.866517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.866849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.867203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.867517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.867909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.868154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.868492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.868813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.869153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.869485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.869812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.870128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.870443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.870767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.871088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.871401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.871755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.872071 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.872412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.872741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.873025 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.873365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.873686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.874001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.874313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.874624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.875484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.875484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.875788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.876030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.876342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO dbcounter [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Registered counter for database nova_api Sep 17 04:27:40.876693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.876693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.877178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.877415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.877640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.877974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.878286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.878600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.878931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.879249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.879579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.879899 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.880255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.880770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.881126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.881708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.882054 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.882391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.882734 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.883073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.883390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.883703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.884050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.884386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.884704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.885071 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.885401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.885722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.886108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.886432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.886751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.887068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.887448 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.887705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.888023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.888342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.888657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.888974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.889317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.889668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.889997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.890347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.890673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.891029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.891352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.891892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.892511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.892847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.893181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.893529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.893863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.894181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.894620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.894946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.895280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.895610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.895937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.896269 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.896600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.896919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.897166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_db.sqlalchemy.engines [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:40.897461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.897818 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.898156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.898682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.899023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.899359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.899688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.899989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.900303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.900621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.900948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.901293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.901605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.902071 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.902239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.902553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.902864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.903178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.903484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.903802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.904112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.904422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.904731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.904814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writer thread running {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:40.905176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.905419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.905689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.905990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.906303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.906695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.907006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.907310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.907630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.907962 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.908303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.908627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.908959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.909300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.909630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.909949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.910277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.910605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.910936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.911252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.911621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.911942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.912266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.912587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.912904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.913254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.913569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.913904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.914236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.914549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.914883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.915209 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.915539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.915860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.916177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.916516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.916834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.917366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.917945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.918278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.918603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.918935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.919256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.919577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.919905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.920231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.920576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.920910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.921270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.921606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.921957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.922330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.922653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.923010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.923348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.923671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.923990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.924303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 04:27:40.924554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 04:27:40.962031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.context [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fa9fccbc-46c5-4ec3-934d-edb50649c947(cell1) {{(pid=120627) load_cells /opt/stack/nova/nova/context.py:464}} Sep 17 04:27:40.966093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:27:40.966544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:27:40.968127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:27:40.968857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:27:40.969356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:27:40.971350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:27:40.973361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO dbcounter [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Registered counter for database nova_cell0 Sep 17 04:27:40.977494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO dbcounter [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Registered counter for database nova_cell1 Sep 17 04:27:40.985524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_db.sqlalchemy.engines [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:40.987208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_db.sqlalchemy.engines [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:40.987348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO dbcounter [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Registered counter for database nova_api Sep 17 04:27:40.992795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writer thread running {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:40.995406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writer thread running {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.007900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_db.sqlalchemy.engines [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:41.015292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writer thread running {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.072550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.context [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),fa9fccbc-46c5-4ec3-934d-edb50649c947(cell1) {{(pid=120626) load_cells /opt/stack/nova/nova/context.py:464}} Sep 17 04:27:41.076587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:27:41.077067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:27:41.078735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:27:41.079449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:27:41.079906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:27:41.081954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:27:41.083961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO dbcounter [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Registered counter for database nova_cell0 Sep 17 04:27:41.087973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO dbcounter [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Registered counter for database nova_cell1 Sep 17 04:27:41.096081 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_db.sqlalchemy.engines [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:41.097944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_db.sqlalchemy.engines [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:41.103861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writer thread running {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.106406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writer thread running {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.139711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO dbcounter [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Registered counter for database nova_cell0 Sep 17 04:27:41.146812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_db.sqlalchemy.engines [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:41.151642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writer thread running {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.253640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO dbcounter [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Registered counter for database nova_cell0 Sep 17 04:27:41.260601 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_db.sqlalchemy.engines [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 04:27:41.265033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writer thread running {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 04:27:41.460834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: WARNING keystonemiddleware.auth_token [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:27:41.468159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: WARNING keystonemiddleware.auth_token [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:27:41.582971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: WARNING keystonemiddleware.auth_token [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:27:41.593825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: WARNING keystonemiddleware.auth_token [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:27:41.807886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: WARNING keystonemiddleware.auth_token [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:27:41.814369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: WARNING keystonemiddleware.auth_token [None req-ca505d1f-d2cf-4539-a561-92882497d661 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:27:41.819806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3f529b9c0 pid: 120627 (default app) Sep 17 04:27:41.936898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: WARNING keystonemiddleware.auth_token [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 17 04:27:41.942982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: WARNING keystonemiddleware.auth_token [None req-d6a0bb60-12b5-4341-b4ac-a187e590bacc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 17 04:27:41.948324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3f529b9c0 pid: 120626 (default app) Sep 17 04:28:18.222196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:18.225505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:18.228873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:18.229875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.260632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5d8c8ba4-e6a9-4317-b89a-ab7a3d3259a4 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Action: 'create', calling method: .version_select at 0x7fbc504f64c0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1349842546", "metadata": {"Type": "work"}}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5d8c8ba4-e6a9-4317-b89a-ab7a3d3259a4 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 122 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:28:20.275110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1349842546", "metadata": {"Type": "work"}}} Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:28:20.276813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1349842546', 'metadata': {'Type': 'work'}}}} Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise error Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi '#$%' Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:20.277868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 17 04:28:20.279542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 17 04:28:20.280871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 17 04:28:20.280871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:20.282870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5d8c8ba4-e6a9-4317-b89a-ab7a3d3259a4 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:20.285705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5d8c8ba4-e6a9-4317-b89a-ab7a3d3259a4 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 2.078961 Sep 17 04:28:20.295969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 1/1] 10.222.0.60 () {60 vars in 1156 bytes} [Sat Sep 17 04:28:18 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 2086 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:28:20.310257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.311537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.314779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:20.317314 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:21.010330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c5d1c6d9-7ad8-4c46-80ea-8010fbf40c1e tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Action: 'create', calling method: .version_select at 0x7fbc505aaf70>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1112769649", "metadata": {"Type": "work"}}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c5d1c6d9-7ad8-4c46-80ea-8010fbf40c1e tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 120 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:28:21.016880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1112769649", "metadata": {"Type": "work"}}} Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: Sep 17 04:28:21.018485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1112769649', 'metadata': {'Type': 'work'}}}} Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise error Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi '0' Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:21.019394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 17 04:28:21.021724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 17 04:28:21.023006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 17 04:28:21.023006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.023006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c5d1c6d9-7ad8-4c46-80ea-8010fbf40c1e tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:21.023006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c5d1c6d9-7ad8-4c46-80ea-8010fbf40c1e tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.719575 Sep 17 04:28:21.023006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 1/2] 10.222.0.60 () {60 vars in 1156 bytes} [Sat Sep 17 04:28:20 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 722 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:28:21.029210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3e2fe8df-002b-4327-94b7-740832cda54c tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Action: 'create', calling method: .version_select at 0x7fbc504f6dc0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1027234418", "metadata": {"Type": "work"}}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3e2fe8df-002b-4327-94b7-740832cda54c tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 119 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:28:21.033828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1027234418", "metadata": {"Type": "work"}}} Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:28:21.035379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1027234418', 'metadata': {'Type': 'work'}}}} Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise error Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi '' Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:21.036206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 17 04:28:21.037627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 17 04:28:21.039077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 17 04:28:21.039077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.039077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3e2fe8df-002b-4327-94b7-740832cda54c tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:21.039077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3e2fe8df-002b-4327-94b7-740832cda54c tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010242 Sep 17 04:28:21.052950 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 2/3] 10.222.0.60 () {60 vars in 1156 bytes} [Sat Sep 17 04:28:21 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 29 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:28:21.057761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ab69d2a2-853a-479f-83d4-1bd00193bf7a tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Calling method '.version_select at 0x7fbc50555820>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:21.702422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-ab69d2a2-853a-479f-83d4-1bd00193bf7a tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] HTTP exception thrown: Volume tempest-invalid-14126793 could not be found. Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ab69d2a2-853a-479f-83d4-1bd00193bf7a tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-14126793 HTTP/1.1 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:28:21.707131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Volume tempest-invalid-14126793 could not be found. Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': 'tempest-invalid-14126793'} Sep 17 04:28:21.708715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-14126793 could not be found. (HTTP 404) (Request-ID: req-ccbeec0c-28fc-4841-afd1-cc2334030578) Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.709736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:28:21.711205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-14126793 could not be found. Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:28:21.712743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-14126793 could not be found. Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-14126793 could not be found. Sep 17 04:28:21.714247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:28:21.715976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ab69d2a2-853a-479f-83d4-1bd00193bf7a tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 404 to user: Volume tempest-invalid-14126793 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:21.717066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ab69d2a2-853a-479f-83d4-1bd00193bf7a tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/tempest-invalid-14126793" status: 404 len: 97 microversion: 2.1 time: 0.666221 Sep 17 04:28:21.718097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1212 bytes} [Sat Sep 17 04:28:21 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-14126793 => generated 97 bytes in 671 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 17 04:28:21.767678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dc208e15-ba3f-4732-b2b6-b666415706cd tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004366 Sep 17 04:28:21.767678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 3/5] 10.222.0.60 () {58 vars in 1140 bytes} [Sat Sep 17 04:28:21 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 17 04:28:21.768532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0cd1cee1-41af-4185-8378-bc5add55bfa9 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004827 Sep 17 04:28:21.768532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 3/6] 10.222.0.60 () {58 vars in 1137 bytes} [Sat Sep 17 04:28:21 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Sep 17 04:28:21.768792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a578636-a801-43a6-831b-cd48c1921973 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Calling method '.version_select at 0x7fbc503b3430>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:22.336187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dcc4bb38-841b-4107-a267-7889c61e35c6 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc503348b0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:22.461242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-7a578636-a801-43a6-831b-cd48c1921973 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] HTTP exception thrown: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a578636-a801-43a6-831b-cd48c1921973 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/os-volumes/f92a6999-0e5d-4606-ace5-b34f3f8c1704 HTTP/1.1 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:28:22.465891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'f92a6999-0e5d-4606-ace5-b34f3f8c1704'} Sep 17 04:28:22.466993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. (HTTP 404) (Request-ID: req-904e0070-997f-42b4-b968-cf9edb039469) Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.467646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:28:22.468464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 17 04:28:22.469317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. Sep 17 04:28:22.470293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.472512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a578636-a801-43a6-831b-cd48c1921973 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 404 to user: Volume f92a6999-0e5d-4606-ace5-b34f3f8c1704 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:22.474392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7a578636-a801-43a6-831b-cd48c1921973 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/f92a6999-0e5d-4606-ace5-b34f3f8c1704" status: 404 len: 109 microversion: 2.1 time: 0.726734 Sep 17 04:28:22.476137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:28:21 2022] DELETE /compute/v2.1/os-volumes/f92a6999-0e5d-4606-ace5-b34f3f8c1704 => generated 109 bytes in 729 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:28:22.492252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67ba1de0-f36e-4e8d-86ff-51dca8277846 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Calling method '.version_select at 0x7fbc5033af70>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:22.659569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dcc4bb38-841b-4107-a267-7889c61e35c6 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/62649680-fa93-40df-bd41-0c8f3216ea28" status: 200 len: 351 microversion: 2.1 time: 0.327275 Sep 17 04:28:22.660336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:28:22 2022] GET /compute/v2.1/os-volumes/62649680-fa93-40df-bd41-0c8f3216ea28 => generated 351 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:28:22.714510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-67ba1de0-f36e-4e8d-86ff-51dca8277846 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] HTTP exception thrown: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67ba1de0-f36e-4e8d-86ff-51dca8277846 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Request method failure captured: Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/os-volumes/97fd2285-5878-4e3a-ae90-8d61b45cdeec HTTP/1.1 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUziGtJYx_oxgmpj4dT1sjMArnVKtXl9TE_mfYHd1_7vIxp_iASKwEXN11MPIeipci-zciGg7xNP0dE-Aoy3-15X7kwXj3l8j0K6Oy0MmdpvQCzc5osauqENmKUFsHlkvRIWX8LxKIb3BcigsqYqYBUOtu6vT5gfD1kisGWybqQ6J1M Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:28:22.721444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/8789a927bb6442da93e546b74955f415"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_8789a927bb6442da93e546b74955f415"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/8789a927bb6442da93e546b74955f415"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 8789a927bb6442da93e546b74955f415 Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesNegativeTest-1591284520 Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 8a4732d60b5441d8bd2f0896d64ca832 Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesNegativeTest-1591284520-project Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '97fd2285-5878-4e3a-ae90-8d61b45cdeec'} Sep 17 04:28:22.722589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. (HTTP 404) (Request-ID: req-ed376da8-efeb-4c4b-a057-f57f6bd1b4bf) Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.723172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:28:22.723989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 17 04:28:22.724980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67ba1de0-f36e-4e8d-86ff-51dca8277846 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] Returning 404 to user: Volume 97fd2285-5878-4e3a-ae90-8d61b45cdeec could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:28:22.726157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-67ba1de0-f36e-4e8d-86ff-51dca8277846 tempest-VolumesNegativeTest-1591284520 tempest-VolumesNegativeTest-1591284520-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/97fd2285-5878-4e3a-ae90-8d61b45cdeec" status: 404 len: 109 microversion: 2.1 time: 0.241753 Sep 17 04:28:22.736385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 5/9] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:28:22 2022] GET /compute/v2.1/os-volumes/97fd2285-5878-4e3a-ae90-8d61b45cdeec => generated 109 bytes in 255 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 17 04:28:25.099264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b242880-43fb-45dc-8936-74376df0065a tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502da040>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:25.208624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5b242880-43fb-45dc-8936-74376df0065a tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/12940fff-0703-4b9e-8a7c-28b0ad9854ea" status: 200 len: 351 microversion: 2.1 time: 0.113485 Sep 17 04:28:25.218537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 5/10] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:28:25 2022] GET /compute/v2.1/os-volumes/12940fff-0703-4b9e-8a7c-28b0ad9854ea => generated 351 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:26.171550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d37d6825-28f3-4796-baa3-a9dd357f3bd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: .version_select at 0x7fbc5033ab80>, body: {"keypair": {"name": "tempest-keypair-272240666", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE357zDWLx8+1GlEaIwOx2UisiRVa5m1C2y0l9WBFBu4DfQ4NmydJkcE5qWRvmeVEBkGjjS1rSkdNh9Vfcb6uW0LsEFh+WTT4f1a+uwD8FLASkTFuf4Sc8RrBNSc2Q47lg=="}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:26.351117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-d37d6825-28f3-4796-baa3-a9dd357f3bd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:28:26.400306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-d37d6825-28f3-4796-baa3-a9dd357f3bd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:28:26.566756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d37d6825-28f3-4796-baa3-a9dd357f3bd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.702121 Sep 17 04:28:26.588379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 6/11] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:28:25 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:27.217917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-216a6d76-ce90-404f-a8f0-1088366fe356 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502da790>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:27.365421 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-216a6d76-ce90-404f-a8f0-1088366fe356 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/db27665e-f863-40b4-ba39-7c25d7dfb809" status: 200 len: 350 microversion: 2.1 time: 0.156517 Sep 17 04:28:27.366815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:28:27 2022] GET /compute/v2.1/os-volumes/db27665e-f863-40b4-ba39-7c25d7dfb809 => generated 350 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:27.384608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-706eaad4-1fb3-493b-aa16-6d289e0f7a20 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502b4700>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:27.598203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-706eaad4-1fb3-493b-aa16-6d289e0f7a20 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.218823 Sep 17 04:28:27.609072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1134 bytes} [Sat Sep 17 04:28:27 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:27.619448 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9a8d725c-8011-423b-8c40-11728768214d tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502dae50>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:27.888895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9a8d725c-8011-423b-8c40-11728768214d tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.278326 Sep 17 04:28:27.889834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 7/14] 10.222.0.60 () {58 vars in 1149 bytes} [Sat Sep 17 04:28:27 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:27.903870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e5b7daf1-de78-4a8c-96f9-8453caa20e79 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc50267af0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:28.116811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e5b7daf1-de78-4a8c-96f9-8453caa20e79 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.215368 Sep 17 04:28:28.117895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 8/15] 10.222.0.60 () {58 vars in 1134 bytes} [Sat Sep 17 04:28:27 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:28.138352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-16a77949-5f73-4f8d-a06e-4da521d84dd9 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502da0d0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:28.353377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-16a77949-5f73-4f8d-a06e-4da521d84dd9 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.225800 Sep 17 04:28:28.354356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1167 bytes} [Sat Sep 17 04:28:28 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:28:28.370937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b5b571f2-4064-4f33-ba50-f7b512588418 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc50b9bdc0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:28.660253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b5b571f2-4064-4f33-ba50-f7b512588418 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.294608 Sep 17 04:28:28.661765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 9/17] 10.222.0.60 () {58 vars in 1170 bytes} [Sat Sep 17 04:28:28 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:28.677604 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a596bd98-d149-4960-9664-47095ac607ed tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc502e6d30>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:28.935591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a596bd98-d149-4960-9664-47095ac607ed tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.263717 Sep 17 04:28:28.936233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 9/18] 10.222.0.60 () {58 vars in 1155 bytes} [Sat Sep 17 04:28:28 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:28.953073 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-58d8728d-337e-40a3-a8c9-de9a16b97792 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc50277af0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:29.210269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-58d8728d-337e-40a3-a8c9-de9a16b97792 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.263722 Sep 17 04:28:29.212116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 10/19] 10.222.0.60 () {58 vars in 1188 bytes} [Sat Sep 17 04:28:28 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:29.240826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6f4e6e48-014a-4132-a939-c61c41aab510 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] Calling method '.version_select at 0x7fbc50334d30>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:29.509223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6f4e6e48-014a-4132-a939-c61c41aab510 tempest-VolumesTestJSON-1202786737 tempest-VolumesTestJSON-1202786737-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.285924 Sep 17 04:28:29.510116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 10/20] 10.222.0.60 () {58 vars in 1155 bytes} [Sat Sep 17 04:28:29 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:30.280158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ae4508ba-5659-417b-9535-ef60dad81788 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: .version_select at 0x7fbc50267e50>, body: {"keypair": {"name": "tempest-keypair-7217238", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHrjQgMBIDCOA+9JevDk/DNpQdckpE/YWRiOIODnHbj0VVhevT4AYr4S+1R5UfVZnqIPnijch/mHBQR9SadUNFPOJwsPZxwuJgZ5x4p/Z0coA7Jji5gf+4kOlpU2NKyd+w=="}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:30.378762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-ae4508ba-5659-417b-9535-ef60dad81788 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for project e0b194b28dec41119ae0d6d913f19c31. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:28:30.393559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-ae4508ba-5659-417b-9535-ef60dad81788 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for user ed4e695113f843fda52ce62792a36bb0 and project e0b194b28dec41119ae0d6d913f19c31. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:28:30.447619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ae4508ba-5659-417b-9535-ef60dad81788 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 201 len: 398 microversion: 2.20 time: 0.451761 Sep 17 04:28:30.451389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 11/21] 10.222.0.60 () {62 vars in 1200 bytes} [Sat Sep 17 04:28:29 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 457 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 17 04:28:33.519778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1811028892", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5d741fbf-1171-42a3-8de2-aef8bbcc1581"}], "security_groups": [{"name": "tempest-securitygroup--317615144"}], "key_name": "tempest-keypair-272240666", "user_data": "}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:33.940728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] validate_networks() for [('5d741fbf-1171-42a3-8de2-aef8bbcc1581', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:28:34.048117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-2140975958", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "networks": [{"uuid": "a004870c-865e-4a45-89cd-2aaecaa38ecd"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:34.280406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] validate_networks() for [('a004870c-865e-4a45-89cd-2aaecaa38ecd', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:28:36.208450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:28:36.208958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:28:36.209497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:28:36.209909 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:28:36.210317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:28:36.222782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:36.223597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:36.224221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:36.225224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:36.226035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:36.226729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:36.293375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:28:36.303153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:28:36.332947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:28:36.336449 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:28:36.423781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:28:36.425443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:28:36.426128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:28:36.426727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:28:36.427338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:28:36.487607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:36.489122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:36.489746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:36.493631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:36.495958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:36.496470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:36.530367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d522f394-1da6-454b-9799-7f1693a2181b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 3.015244 Sep 17 04:28:36.531009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 12/22] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:28:33 2022] POST /compute/v2.1/servers => generated 401 bytes in 3021 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 17 04:28:36.554263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1996749557", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e95cd093-c909-4fa1-980b-df0004dd3cf5"}], "security_groups": [{"name": "tempest-securitygroup--1297374440"}], "key_name": "tempest-keypair-7217238", "user_data": "}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:36.560040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Getting quotas for project 39cc778e84014ed98f33554b7c46b174. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:28:36.568614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Getting quotas for user 9be671f062ed4cb793facf9ca7920263 and project 39cc778e84014ed98f33554b7c46b174. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:28:36.584972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:28:36.588252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:28:36.745974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8ffd1a24-5126-4b93-a3ae-1ad4cea677b8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.413899 Sep 17 04:28:36.747940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 11/23] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:28:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 3416 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:28:36.758903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-54f2623d-765d-41c6-bef8-8b05fcee85b1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:36.762522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-54f2623d-765d-41c6-bef8-8b05fcee85b1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:36.918988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] validate_networks() for [('e95cd093-c909-4fa1-980b-df0004dd3cf5', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:28:37.014622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-54f2623d-765d-41c6-bef8-8b05fcee85b1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:37.039934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-54f2623d-765d-41c6-bef8-8b05fcee85b1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1241 microversion: 2.1 time: 0.290978 Sep 17 04:28:37.041651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 12/24] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:36 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1241 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:37.047724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-62226111-4a63-428b-b78a-9238864ccca7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:37.048565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-62226111-4a63-428b-b78a-9238864ccca7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:37.137007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-62226111-4a63-428b-b78a-9238864ccca7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:37.148126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-62226111-4a63-428b-b78a-9238864ccca7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1215 microversion: 2.1 time: 0.105007 Sep 17 04:28:37.163830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 13/25] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:37 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1215 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:37.941298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:28:37.941833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:28:37.942445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:28:37.943050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:28:37.943471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:28:37.953653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:37.954191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:37.954676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:37.955571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:37.956028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:37.956557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:37.982668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for project e0b194b28dec41119ae0d6d913f19c31. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:28:37.988524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for user ed4e695113f843fda52ce62792a36bb0 and project e0b194b28dec41119ae0d6d913f19c31. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:28:38.009630 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:28:38.015481 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:28:38.074603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-78adea0a-1728-40a3-a189-b7db11146deb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:38.076598 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-78adea0a-1728-40a3-a189-b7db11146deb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:38.127390 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-39520af6-0208-4abc-8694-6e61f7e729b3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.594262 Sep 17 04:28:38.136894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 13/26] 10.222.0.60 () {62 vars in 1188 bytes} [Sat Sep 17 04:28:36 2022] POST /compute/v2.1/servers => generated 402 bytes in 1605 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 17 04:28:38.142639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:38.143569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:38.221404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-78adea0a-1728-40a3-a189-b7db11146deb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:38.229207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-78adea0a-1728-40a3-a189-b7db11146deb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1241 microversion: 2.1 time: 0.162578 Sep 17 04:28:38.250517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 14/27] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:38 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1241 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:38.274538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-be220037-474e-4467-a578-46bcfee8007b tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:38.277802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-be220037-474e-4467-a578-46bcfee8007b tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:38.434389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:38.448469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-be220037-474e-4467-a578-46bcfee8007b tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:38.462378 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-be220037-474e-4467-a578-46bcfee8007b tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1215 microversion: 2.1 time: 0.204816 Sep 17 04:28:38.464087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:38.468875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:38.470133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6fed3cdc-a88c-4866-9d03-be09ffd33fa2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1279 microversion: 2.20 time: 0.331859 Sep 17 04:28:38.482277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 15/28] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:38 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1215 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:38.497214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 14/29] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:38 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1279 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:39.258257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:39.258850 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:39.273724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:39.274333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:39.274791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:39.275546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:39.276015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:39.282715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:39.489715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ce2e7491-01e4-4bf5-8989-e96f6df897d9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:39.490515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ce2e7491-01e4-4bf5-8989-e96f6df897d9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:39.761035 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:39.782695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-04274115-7820-4835-a231-2f7cb0a2f451 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1237 microversion: 2.1 time: 0.528327 Sep 17 04:28:39.783777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 16/30] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:39 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1237 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:39.790863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:39.791684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:39.798525 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ce2e7491-01e4-4bf5-8989-e96f6df897d9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:39.810795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ce2e7491-01e4-4bf5-8989-e96f6df897d9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1215 microversion: 2.1 time: 0.325748 Sep 17 04:28:39.825345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 15/31] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:39 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1215 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:39.992816 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:40.004082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:40.009682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:40.012433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f604eb78-e9f5-4bc9-9aa5-44e4c1af2b5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1279 microversion: 2.20 time: 0.225903 Sep 17 04:28:40.033654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 17/32] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:39 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1279 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:40.808327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:40.809438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:40.821152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:40.823401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:40.824465 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:40.825879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:40.827358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:40.828433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:40.838785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:40.843491 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:40.864665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:40.865786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:40.866966 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:40.872266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:40.874134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:40.874973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:41.009516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:41.020357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2cafc4ce-b14b-4ed3-9abd-b6b24a7174ef tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1267 microversion: 2.1 time: 0.186833 Sep 17 04:28:41.039225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 18/33] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:40 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1267 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:28:41.049774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:41.050825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:41.062808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:41.064084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:41.067028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:41.068232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:41.068959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:41.069665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:41.185179 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:41.196679 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:41.199349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:41.201470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3b9fd3cb-8a3a-406c-9954-6c8a48bb428d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1275 microversion: 2.20 time: 0.155742 Sep 17 04:28:41.202618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 19/34] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:41 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1275 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:41.222150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:41.238034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c3ad744b-669d-4ac5-9835-fbeb2315d1cd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1299 microversion: 2.1 time: 0.434141 Sep 17 04:28:41.239058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 16/35] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:40 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1299 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:42.053549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:42.054734 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:42.069460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.070661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.071122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:42.074486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.075197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.076198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:42.232389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:42.233632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:42.258452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.259486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.264519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:42.266602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.270053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.273883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:42.592696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:42.605075 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:42.619618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:42.622633 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dd8a0774-e56a-47cc-83b1-512ed3255417 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1339 microversion: 2.20 time: 0.394388 Sep 17 04:28:42.649918 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 17/36] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:42 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1339 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:28:42.670151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:42.671922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:42.699400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.701270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.702092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:42.708129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:42.709261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:42.710363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:43.438153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:43.464064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6627ddb5-7fd0-467e-a8ec-a2d6ec2b88e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1273 microversion: 2.1 time: 1.410113 Sep 17 04:28:43.468926 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 20/37] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:42 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1273 bytes in 1422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:43.662290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:43.663243 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:43.683692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:43.684207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:43.684678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:43.685440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:43.685901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:43.686349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:43.745382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:43.762665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6c81abaf-7400-4349-950e-301f8d3d9a87 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 1.107497 Sep 17 04:28:43.790035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 18/38] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:42 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 1139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:43.985875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:43.998293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:44.002695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:44.007553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-298bb376-fdb3-4340-b12e-13ca2b45f435 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1337 microversion: 2.20 time: 0.350079 Sep 17 04:28:44.012005 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 21/39] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:43 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1337 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:28:44.497539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:44.498505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:44.524799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:44.525393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:44.525834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:44.526564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:44.527017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:44.527478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:44.801826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:44.804837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:44.822059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:44.822718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:44.825309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:44.828665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:44.833582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:44.836737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-795abf3b-1ae2-4606-bd1e-471782997713 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1273 microversion: 2.1 time: 0.345298 Sep 17 04:28:44.837398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:44.839313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:44.868536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 19/40] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:44 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1273 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:45.038298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:45.040049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:45.052149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:45.053470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:45.054764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:45.056084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:45.057217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:45.058171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:45.320139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:45.335703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:45.344744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:45.351949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3780f7f1-95ff-48b2-ab4e-a1b5eb4e49bc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1337 microversion: 2.20 time: 0.316934 Sep 17 04:28:45.370541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 20/41] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:45 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1337 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:45.845181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:45.856002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9760e1d5-25bc-432b-925f-711f2092a7c7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 1.059523 Sep 17 04:28:45.859600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 22/42] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:44 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:28:45.900831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:45.900831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:45.900831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:45.900831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:45.902460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:45.902460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:45.902460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:45.902460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:46.388806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:46.389446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:46.400020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:46.414699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:46.417945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:46.421913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:46.426694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:46.428011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:46.539142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:46.589639 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:46.611606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:46.617333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:46.619761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5ad93524-cd0d-4998-b5e1-29d19093de19 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1337 microversion: 2.20 time: 0.234786 Sep 17 04:28:46.622569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 23/43] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:46 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1337 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:28:46.892673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:46.893316 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:46.905057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:46.905599 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:46.906054 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:46.906850 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:46.912400 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:46.913174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:46.925000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:46.935782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2532f1a8-a666-42e9-bab0-9fe27bac11a7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 1.058054 Sep 17 04:28:46.955715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 21/44] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:45 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 1081 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:47.554948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:47.569992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc5b62e3-c347-47f7-a925-cdbe032a2751 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.681808 Sep 17 04:28:47.570584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 24/45] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:46 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:28:47.654404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:47.655456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:47.662716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:47.663673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:47.664262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:47.665344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:47.666205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:47.666830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:47.965487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:47.966718 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:47.967024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=4 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:47.983828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:47.985136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:47.986012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:47.987303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:47.988211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:47.989184 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.257270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:48.257952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:48.279228 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1ddb0be5-e0f4-448e-a67f-ebb1a7f6d835 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.319951 Sep 17 04:28:48.283288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 25/46] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:47 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:48.284767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:48.287551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:48.289390 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-603a21cc-450c-41fd-a09b-16bb2ecd830c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.642050 Sep 17 04:28:48.303059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 22/47] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:47 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:48.596027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:48.596657 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:48.600043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=8 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:48.604737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:48.605265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:48.605702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.606399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:48.606861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:48.607296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.664848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "508f0590-1d24-42b5-9275-280c6cedff50", "tag": "976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:48.719632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:48.719632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:48.719632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.771404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:48.780890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-43003292-0d8a-4372-9df2-93d5fd96d2d3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.188818 Sep 17 04:28:48.781895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 23/48] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:48 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:48.789111 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Creating event network-changed:976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba for instance 508f0590-1d24-42b5-9275-280c6cedff50 on hv-cinder-85799 Sep 17 04:28:48.789946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120626) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 17 04:28:48.791567 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b49a93ee-1e1a-4a38-b05b-5d196f3ce126 req-72c48434-9085-4f71-9676-942d2af2da9a service nova] Acquired lock "compute-rpcapi-router" {{(pid=120626) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 17 04:28:48.794920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:48.797863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:48.798572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.799512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:48.800151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:48.800898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:48.811426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.compute.rpcapi [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 17 04:28:48.812769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] Releasing lock "compute-rpcapi-router" {{(pid=120626) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 17 04:28:48.831230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8b609e3b-bf07-485d-af21-25be9e99d805 None None] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.546556 Sep 17 04:28:48.831891 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 26/49] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:28:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:49.308206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:49.309946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:49.323404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.328854 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.329587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.331215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.332547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.333291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.438890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092", "tag": "2666bef0-34b1-4359-b664-5517c071bcec"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:49.458468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.458953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.459384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.511027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Creating event network-changed:2666bef0-34b1-4359-b664-5517c071bcec for instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 on hv-cinder-85799 Sep 17 04:28:49.517912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120627) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 17 04:28:49.519248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-79f362ad-a061-43eb-a166-795ecaac891d req-0fe5e72f-ad2b-4448-8df8-647c7977c635 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120627) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 17 04:28:49.520412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.522291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.523558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.525483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.526861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.528691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.551296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.compute.rpcapi [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 17 04:28:49.554544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] Releasing lock "compute-rpcapi-router" {{(pid=120627) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 17 04:28:49.619861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d421ce64-cd3d-4180-a3ae-f3d517c77812 None None] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.502524 Sep 17 04:28:49.620825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 24/50] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:28:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:49.628568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:49.633923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:49.642641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4697aaff-8ed3-44cc-a6c4-beee27aca778 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.338794 Sep 17 04:28:49.643628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:49.650092 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 27/51] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:49 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:49.655088 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.655812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.656415 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.657850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.658457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.659051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.805754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=24 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:49.806843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:49.807686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:49.822100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.823546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.825405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.827086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:49.828498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:49.829525 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:49.996397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:50.003476 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=16 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:50.006586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:50.008915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:50.010530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-84468d7c-e989-4277-a8b5-37ce6748ecd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.388542 Sep 17 04:28:50.012505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 25/52] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:49 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:50.138048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:50.148632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-398a1160-ba58-4af1-ad58-ac47ebe0d233 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.347136 Sep 17 04:28:50.149255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 28/53] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:49 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:50.671365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:50.672927 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:50.687586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:50.697497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:50.698172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:50.700902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:50.702512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:50.703154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:51.011017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:51.024623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-45fb8f30-91fd-4949-8da0-e68083aafef0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.358442 Sep 17 04:28:51.026165 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 26/54] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:50 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:51.037231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:51.037880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:51.050301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:51.051516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:51.051516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:51.051955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:51.052443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:51.053112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:51.172484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:51.173128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:51.183332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:51.183841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:51.184266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:51.184983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:51.200890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:51.201911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:51.413341 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:51.436453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:51.438924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:51.465920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-073a7183-8313-44e4-a6da-1c95f099f356 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.431308 Sep 17 04:28:51.468519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 29/55] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:51 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:51.509125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:51.518489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f8425ba7-8aff-4ba3-a40d-6c0bbb0542b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.350319 Sep 17 04:28:51.519679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 27/56] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:51 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:52.051886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:52.057232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:52.067782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.070207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.071332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.072940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.075141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.076488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.255689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:52.264789 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-12aa67a2-faa7-42c0-bc13-541aad05198c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.216753 Sep 17 04:28:52.268285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 30/57] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:52 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:52.500925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:52.502309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:52.511902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.515078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.515531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.516474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.516933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.517405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.546621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:52.547800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:52.565399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.566322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.566938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.573546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:52.574232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:52.574832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:52.850952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:52.861571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc8957bd-8f9d-408a-8e53-8d307136e548 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.318815 Sep 17 04:28:52.865305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 31/58] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:52 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:52.886828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:52.935914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:52.945703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:52.946899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f2f0f29a-154a-4a6f-857c-828e139331bf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.451604 Sep 17 04:28:52.969911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 28/59] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:52 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:53.292540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:53.293182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:53.322851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:53.323817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:53.324266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:53.332660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:53.338156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:53.338610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:53.901438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:53.902198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:53.930622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:53.933043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:53.934424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:53.935456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:53.942096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:53.949907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:54.519953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:54.539035 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b96ea3d9-55b8-434a-8bb4-d6c09b3bd7e5 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 1.250887 Sep 17 04:28:54.542908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 32/60] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:53 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 1255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:54.545311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:54.553988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:54.560311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:54.562560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:54.563696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:54.564720 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:54.565717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:54.566871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c3c9dbf5-0873-4d88-91f3-df859b91fddf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.670890 Sep 17 04:28:54.569606 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 29/61] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:53 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:54.589988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:54.591782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:54.617820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:55.001783 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:55.018252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:55.022500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:55.025407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ce0d5271-4313-4ccf-ae36-5a387fd4b30d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.480914 Sep 17 04:28:55.028345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 33/62] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:54 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:55.566850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:55.568283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:55.580042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:55.586050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:55.586731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:55.587957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:55.588687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:55.589494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:55.592500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:55.593122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:55.604051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:55.609431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:55.610133 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:55.611033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:55.611671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:55.612246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:55.856702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:55.868949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7c8f4379-4488-4367-8b10-8a712498375b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.280102 Sep 17 04:28:55.870266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 34/63] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:55 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:55.878149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:28:55.890418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:55.891440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:55.892116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:55.907435 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] Creating event network-changed:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:28:55.920206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-49a53fb5-1d32-4b61-a3ef-e289baf79245 req-5d7eca0c-fc57-4612-b78f-6d69f006299b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048247 Sep 17 04:28:55.924888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 35/64] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:28:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:28:56.057827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:56.059706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:56.076958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:56.077468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:56.077959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:56.078735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:56.079208 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:56.080249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:56.086066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:56.094254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cbf0ccff-1f7f-4cee-942d-4b6717501666 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.532539 Sep 17 04:28:56.113279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 30/65] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:55 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:56.319182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:56.333266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:56.338299 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:56.340296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1bb2e721-4483-47fe-993e-bd764e3a29de tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.290730 Sep 17 04:28:56.342051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 36/66] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:56 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:56.898442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:56.900025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:56.907520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:56.908180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:56.912196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:56.912196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:56.912196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:56.912196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:57.065914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:57.076360 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-93c5cae0-5b9f-4386-b544-9f9b2b0d73d4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.185154 Sep 17 04:28:57.077252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 31/67] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:56 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:57.134791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:57.135544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:57.142430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:57.143005 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:57.143512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:57.144656 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:57.145860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:57.146790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:57.310749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:57.319574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b0582a00-b7d7-49c3-8907-0019ca1a55e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.189250 Sep 17 04:28:57.320454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 37/68] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:57 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:57.368786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:57.369956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:57.376490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:57.376977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:57.377446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:57.378151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:57.380004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:57.380303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:57.390270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:57.536878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:57.549117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:57.550667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:57.551779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2fd1b795-4591-4e81-8dd1-383a626ebbd1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.187970 Sep 17 04:28:57.552333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 32/69] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:57 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:58.102278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:58.103535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:58.111483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.112275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.113068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.113853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.118641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.119119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.310220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:58.320256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1dfe7fa5-d3bd-4261-8a81-b5feed6db9d8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.223229 Sep 17 04:28:58.320866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 38/70] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:58 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:58.348640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:58.350873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:58.379710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.380214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.384608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.386157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.386626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.387057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.566372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:58.577207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2c0d60db-1c24-4518-b9ad-bf38b7a370d4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.238524 Sep 17 04:28:58.578105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 33/71] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:58 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:28:58.592835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:58.594003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:58.600593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.601214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.602257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.603520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:58.604159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:58.604946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:58.767948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:58.777469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:58.779088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:58.780544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-82b33b55-e2e1-49d2-88c7-fe44e05325f3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.208270 Sep 17 04:28:58.781405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 39/72] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:58 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:28:59.344755 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:59.345620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:59.351748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.352710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.353205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.353914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.354367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.354791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.543497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:59.560482 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:59.568406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4c14c30c-c124-4819-98b8-4929aab273e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.228249 Sep 17 04:28:59.569133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 34/73] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:59 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:59.600266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:59.600838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:59.603245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:28:59.608764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.609417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.609967 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.610741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.613439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.613894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.765542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:59.775882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3c33b079-45bf-41af-b056-c709aa4196e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.179635 Sep 17 04:28:59.777128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 40/74] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:28:59 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:28:59.805305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:28:59.806007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:28:59.812273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.812898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.813358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.814049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:28:59.814500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:28:59.814933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:28:59.986543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:59.994610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:59.996134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:28:59.997623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-517bdccb-a78a-405d-b075-9c565b93cc61 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.196266 Sep 17 04:28:59.998349 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 35/75] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:28:59 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:00.592638 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:00.593277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:00.602454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:00.604073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:00.604652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:00.605529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:00.606121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:00.606714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:00.799959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:00.801147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:00.807697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:00.808346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:00.808947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:00.809838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:00.810429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:00.811015 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:00.889739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:00.897875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-15e4964b-7096-489d-84e3-9824a6d4cb86 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.309921 Sep 17 04:29:00.898446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 41/76] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:00 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:01.014852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:01.022447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:01.023061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:01.029186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3c469811-522d-4c15-aa0b-394fecc408e0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.234132 Sep 17 04:29:01.029672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:01.030277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:01.030638 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:01.030869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 36/77] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:00 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:01.033874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:01.035156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:01.035800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:01.250629 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:01.259284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:01.260645 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:01.261820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7c5a4942-db31-4f95-8dfc-e0e779ef22ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.243598 Sep 17 04:29:01.262443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 42/78] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:01 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:29:01.923041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:01.924529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:01.930963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:01.931455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:01.931886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:01.932578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:01.933026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:01.933469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:02.051825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:02.052423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:02.058525 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:02.059007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:02.059431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:02.060094 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:02.060541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:02.060949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:02.066714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:02.074736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-038f81c2-1eea-42c8-86dd-ccdc359f8cb6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.155910 Sep 17 04:29:02.075312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 37/79] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:01 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:02.187618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:02.195596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-78a6ba35-f1d4-41d6-a55b-142207e7b3e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.147652 Sep 17 04:29:02.196163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 43/80] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:02 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:02.300198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:02.300812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:02.307832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:02.308317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:02.308747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:02.309466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:02.309915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:02.310497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:02.444093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:02.455001 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:02.456445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:02.457787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-974a99f6-658e-41b4-9b11-927f35461bff tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.173752 Sep 17 04:29:02.458538 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 38/81] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:02 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:03.097930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:03.098901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:03.105485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.105968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.106403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.107085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.107722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.108143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.222010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:03.223335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:03.232120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.232520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.233020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.234230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.234791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.235244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.254323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:03.267875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2fb78b4-ac67-4e36-89a8-d33abfba8ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.173591 Sep 17 04:29:03.269541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 44/82] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:03 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:03.416326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:03.425298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-47d65711-fda8-45eb-b957-52b7975791d1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.210229 Sep 17 04:29:03.445177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 39/83] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:03 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:03.482835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:03.483438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:03.485345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:03.490444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.490934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.491368 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.492059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:03.492597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:03.493020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:03.936201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:03.944426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:03.945870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:03.946984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c3bb8038-888f-47bd-9bf6-1cf7b31bcdf7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.468094 Sep 17 04:29:03.947538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 45/84] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:03 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:04.291828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:04.292396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:04.298305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.298956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.299377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:04.300053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.300509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.300919 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:04.432803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:04.441172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3c17591d-2245-48d5-9586-782c371aff83 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.153213 Sep 17 04:29:04.441741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 40/85] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:04 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:04.450003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:04.450669 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:04.457401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.457871 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.458300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:04.458972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.459443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.459860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:04.587052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:04.594914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d8abdf7c-d8dd-45cf-a7da-6a33d3bf68af tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.149491 Sep 17 04:29:04.595377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 46/86] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:04 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:04.971394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:04.972005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:04.978389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.978861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.979298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:04.979974 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:04.980431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:04.980851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:05.116905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:05.125289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:05.126808 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:05.127916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a7302676-63c2-4825-be5e-29d11d82c799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.160561 Sep 17 04:29:05.128473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 41/87] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:04 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:05.464155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:05.464876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:05.471524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:05.472007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:05.472430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:05.473117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:05.473578 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:05.473990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:05.618534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:05.619130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:05.625706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:05.626505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:05.627133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:05.627991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:05.628616 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:05.629248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:05.639837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:05.654560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-04970253-3c0d-4f6a-b8b0-9d2dcaf56cd2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.194338 Sep 17 04:29:05.655309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 47/88] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:05 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:05.784808 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:05.802687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d2743026-edf9-44aa-8f06-655c091205e9 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.181113 Sep 17 04:29:05.802687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 42/89] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:05 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:06.151801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:06.152592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:06.158847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.159328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.160451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.161172 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.161631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.162049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.293239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:06.301987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:06.303424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:06.304523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-82a3c35c-9fce-4dd1-bab7-5e3db87562ba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.156669 Sep 17 04:29:06.305247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 48/90] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:06 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:06.678362 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:06.678941 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:06.685325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.686473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.686923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.687610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.688057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.688510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.819238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:06.819962 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:06.826050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.826797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.827433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.828345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:06.829072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:06.830002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:06.842911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:06.853972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-eddd89b0-f136-4d56-b66a-be813808519b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.179944 Sep 17 04:29:06.854817 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 43/91] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:06 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:06.960955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:06.968794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e6fb69c2-eb0d-4bb4-a3f8-ed79b98a781a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.153658 Sep 17 04:29:06.969424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 49/92] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:06 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:07.333339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:07.334786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:07.345434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:07.345909 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:07.346330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:07.347008 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:07.347450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:07.347862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:07.478593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:07.487176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:07.488647 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:07.489794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6629457a-0060-48c5-8294-fa2c45f85785 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.163242 Sep 17 04:29:07.490385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 44/93] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:07 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:07.878799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:07.879414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:07.885559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:07.886049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:07.886482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:07.887171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:07.887623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:07.888043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:07.991757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:07.992307 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:07.998900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:07.999374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:07.999798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:08.000471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:08.001041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:08.001520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:08.015725 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:08.026150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:08.031798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1f0cbb16-8991-4166-8023-f2dac97561eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.157620 Sep 17 04:29:08.032432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 50/94] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:07 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:08.143355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:08.151126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9fc79775-e486-4a8f-81a5-2d6af1fe76e1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.163614 Sep 17 04:29:08.151691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 45/95] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:07 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:08.515017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:08.515685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:08.522057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:08.522533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:08.523162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:08.523608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:08.524198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:08.524585 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:08.662951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:08.672047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:08.673524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:08.674618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-60e2befd-281f-496c-a813-2667825be02c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.163875 Sep 17 04:29:08.675198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 51/96] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:08 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:09.058911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:09.059932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:09.070697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.071536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.072254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.073435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.074160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.074959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.173878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:09.174478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:09.180811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.181256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.181685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.182344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.182813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.183231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.210209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:09.223822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1bc692a4-3df1-4d02-85ab-b73e9c5bc684 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.171383 Sep 17 04:29:09.224622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 46/97] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:09 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:09.311110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:09.319057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5101be94-985d-48ce-a906-2a9e02655b86 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.149351 Sep 17 04:29:09.319618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 52/98] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:09 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:09.699504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:09.700110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:09.706694 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.707201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.707627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.708304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:09.708745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:09.709195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:09.836657 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:09.845259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:09.846702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:09.847851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-97fb0fc6-0535-49aa-9cc5-f630c590ecbc tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.152322 Sep 17 04:29:09.848388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 47/99] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:09 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:10.249930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:10.250722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:10.259090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.259749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.260335 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:10.261390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.262014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.262593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:10.342037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:10.343011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:10.349419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.349922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.350321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:10.351261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.352060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.352484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:10.399381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:10.407103 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3a7d098c-0d48-4efc-b209-e864b4b87a13 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.162422 Sep 17 04:29:10.407671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 53/100] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:10 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:10.512994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:10.516822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:10.521912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-81eae3f8-9ebc-42e6-b01b-c51e1ce6ae4c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.183863 Sep 17 04:29:10.522502 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 48/101] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:10 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:10.875950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:10.876559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:10.882872 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.883359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.883795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:10.884505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:10.884952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:10.885420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:11.029451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:11.038237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:11.039892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:11.041137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-48511eab-31d6-474a-9def-b47383309c9f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.171660 Sep 17 04:29:11.041667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 54/102] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:10 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:11.430651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:11.431287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:11.438355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:11.438883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:11.439354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:11.440124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:11.440618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:11.441076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:11.544758 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:11.545359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:11.551360 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:11.551839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:11.552264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:11.552946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:11.553419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:11.553831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:11.584380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:11.592446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9a253bd9-98f1-441b-acb6-de01b6c2e7a0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.166407 Sep 17 04:29:11.593028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 49/103] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:11 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:11.682605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:11.690759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5be12d2d-ddca-472d-a5fa-fef09c0d0a71 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.149876 Sep 17 04:29:11.691301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 55/104] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:11 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:12.071985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:12.072671 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:12.079478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.079960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.080388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.081107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.081557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.082000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.215301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:12.223563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:12.224996 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:12.226124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0c0e0845-737c-423d-b351-e6bde7f8b117 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.158141 Sep 17 04:29:12.227053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 50/105] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:12 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:12.616095 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:12.616676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:12.623795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.624449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.625029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.625935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.626541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.627056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.714567 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:12.715154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:12.721492 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.721901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.722501 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.722986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:12.723496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:12.723830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:12.775094 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:12.783120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2356dd8c-83ba-4fa5-9e47-4944fc8ec41d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.170974 Sep 17 04:29:12.783722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 56/106] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:12 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:12.860924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:12.869332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d0a3dd7c-7df6-4e5f-9e94-da1c112fcfad tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.158631 Sep 17 04:29:12.869907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 51/107] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:12 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:13.251202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:13.251807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:13.257963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.258450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.258876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.259557 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.260010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.260428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.389821 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:13.398626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:13.400247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:13.401389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-94c1eea1-bac5-4782-9a22-4c0da2ed9ec5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.154266 Sep 17 04:29:13.401974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 57/108] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:13 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:13.810500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:13.811505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:13.818673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.819160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.819595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.820282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.820731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.821158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.892140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:13.892781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:13.899149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.899693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.900050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.900741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:13.901222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:13.901641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:13.950142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:13.958170 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e807207c-d2ed-4029-b7b7-c9680e49a2f7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.154311 Sep 17 04:29:13.958744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 52/109] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:13 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:14.038510 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:14.046582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-12ce50a3-d354-4134-9991-c9bd77ec87fd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.158418 Sep 17 04:29:14.047157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 58/110] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:13 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:14.425945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:14.426917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:14.433898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:14.434798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:14.435443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:14.436361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:14.437545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:14.438204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:14.599722 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:14.613883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:14.615675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:14.617242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-63459c5a-1c46-4e1a-acf6-1611d54e2877 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.195152 Sep 17 04:29:14.618246 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 53/111] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:14 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:14.672958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "508f0590-1d24-42b5-9275-280c6cedff50", "name": "network-vif-plugged", "status": "completed", "tag": "976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:14.684129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:14.684566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:14.685244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:14.697851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] Creating event network-vif-plugged:976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba for instance 508f0590-1d24-42b5-9275-280c6cedff50 on hv-cinder-85799 Sep 17 04:29:14.720958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-10fd1a71-1888-41ab-b880-eb1f6ccb0527 req-8e238867-e8b8-4037-9929-db9145c35809 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.130223 Sep 17 04:29:14.721956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 59/112] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:29:14.981786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:14.982443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:14.989054 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:14.989574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:14.989980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:14.990665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:14.991111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:14.991518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:15.070226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:15.071150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:15.077697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:15.078178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:15.078607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:15.079331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:15.079780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:15.080196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:15.122499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:15.130439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0807307a-2c6f-4de5-b7b4-d17c6ea94b56 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.152602 Sep 17 04:29:15.131021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 54/113] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:14 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:15.212162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:15.219880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ef9460d5-7980-4935-94c4-74cc80c6a460 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.154005 Sep 17 04:29:15.220440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 60/114] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:15 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:15.646691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:15.647947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:15.657005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:15.658262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:15.658880 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:15.659754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:15.660354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:15.660924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:15.814335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:15.822656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:15.824774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:15.826163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7a56fa2c-8fc4-435d-abb4-edd71a0351f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.184693 Sep 17 04:29:15.839789 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 55/115] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:15 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:16.158484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:16.159484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:16.169662 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.170454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.171157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.172253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.172988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.173716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.243442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:16.243997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:16.251006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.251487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.251915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.252596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.253042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.253500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.317272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:16.325428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3cc4ddbf-5e2e-4d02-99e5-aaf678757d9e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.173566 Sep 17 04:29:16.326019 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 61/116] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:16 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:16.382878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:16.394174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d78aa651-4e4c-42a3-bf6e-f28037bd27df tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.155016 Sep 17 04:29:16.394763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 56/117] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:16 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:16.850898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:16.851874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:16.858036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.858515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.858941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.859611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:16.860056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:16.860470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:16.994294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:17.002831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:17.004253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:17.005376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8c93746e-c9af-448b-933a-da5e98d3f32a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.158392 Sep 17 04:29:17.005918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 62/118] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:16 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:17.352631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:17.353687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:17.362513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:17.362990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:17.363413 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:17.364084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:17.364527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:17.364936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:17.415961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:17.416635 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:17.423124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:17.423617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:17.424042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:17.424728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:17.425223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:17.425646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:17.495746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:17.503499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f4b05f3f-9b7f-4889-a9d6-bcce83ac6f15 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.157668 Sep 17 04:29:17.504043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 57/119] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:17 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:17.556226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:17.564506 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2106af54-083a-4a4f-9a3d-952e108e8d4f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.152380 Sep 17 04:29:17.565053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 63/120] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:17 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:18.042325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:18.043000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:18.050701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.051205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.051687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.052719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.053328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.054212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.190611 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:18.199119 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:18.200549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:18.201710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cf557949-1b54-4699-b825-a36f5dc49d90 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.175858 Sep 17 04:29:18.202270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 58/121] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:18 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:18.530101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:18.531021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:18.537991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.538471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.538899 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.539581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.540044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.540444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.587354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:18.587958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:18.594370 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.594904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.595333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.596019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:18.596468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:18.596921 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:18.670844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:18.678942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fdf9b307-a44e-46e1-b0e7-9a4bb9610a9a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.155516 Sep 17 04:29:18.681114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 64/122] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:18 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:18.723037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:18.731266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2d490aa4-966c-4507-be60-89428736e7dd tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.147960 Sep 17 04:29:18.731842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 59/123] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:18 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:19.231214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:19.232069 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:19.242172 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.243009 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.243741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.244847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.245744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.246357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.377528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:19.388684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:19.390942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:19.392592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-432c4b47-31d3-455b-ba58-422dfff3227f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.168200 Sep 17 04:29:19.393344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 65/124] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:19 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:19.704479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:19.705116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:19.707911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=74 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:19.715006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.715462 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.716042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.717041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.717527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.717952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.754139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:19.754740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:19.761041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.761557 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.761990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.762705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:19.763286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:19.763704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:19.858135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:19.866103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-60c68d02-11ce-4ac5-9145-add233eb9c45 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.167865 Sep 17 04:29:19.866719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 60/125] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:19 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:19.902674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:19.910458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d85a45cf-5c15-4ceb-8201-75ac7cbc2ba7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.160230 Sep 17 04:29:19.911012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 66/126] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:19 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:20.417669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:20.418277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:20.424277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.424756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.425209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:20.425897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.426342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.426759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:20.558927 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:20.567126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:20.568562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:20.569693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-060e69d5-c5aa-4bf8-8709-2070f35845b7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.155949 Sep 17 04:29:20.570244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 61/127] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:20 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:20.889364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:20.890161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:20.897099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.897587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.898029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:20.898705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.899162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.899628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:20.933030 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:20.933832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:20.939932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.940417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.941041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:20.941762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:20.942210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:20.942627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:21.033598 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:21.042329 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-613daf40-e550-443a-9009-4ac7e28ae11a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.157359 Sep 17 04:29:21.042901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 67/128] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:20 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:21.093533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:21.102550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fa0d1f1d-d92c-40cf-8a32-bd455445d067 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.173239 Sep 17 04:29:21.103503 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 62/129] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:20 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:21.594490 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:21.595271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:21.602794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:21.603295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:21.603723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:21.604412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:21.605271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:21.605702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:21.610530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:21.706103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092", "name": "network-vif-plugged", "status": "completed", "tag": "2666bef0-34b1-4359-b664-5517c071bcec"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:21.718846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:21.719309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:21.721364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:21.735030 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] Creating event network-vif-plugged:2666bef0-34b1-4359-b664-5517c071bcec for instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 on hv-cinder-85799 Sep 17 04:29:21.741298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-fe88ec57-34b4-4b6a-b66a-6296886fab30 req-14a546e7-f290-4fff-9d74-b80c13b1929d service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043447 Sep 17 04:29:21.742614 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 63/130] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:29:21.781025 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:21.790112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:21.791786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:21.792896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a8513416-1bb6-4109-9e46-8dbd0eb7421f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.202767 Sep 17 04:29:21.793610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 68/131] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:21 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:22.067465 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:22.068167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:22.074829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.075314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.075741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.076423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.076879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.077341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.124953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:22.125665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:22.131910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.132466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.132975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.133773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.134306 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.134760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.213376 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:22.229317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-867a2ace-8c2c-43b1-a51c-cfaf582612b9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.159638 Sep 17 04:29:22.229317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 64/132] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:22 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:22.271347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:22.281995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc4ce9e9-c798-4737-b5af-76b0a6dbcd7f tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.160676 Sep 17 04:29:22.283165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 69/133] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:22 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:22.818931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:22.820162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:22.827842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.828385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.829043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.829964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:22.830432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:22.831553 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:22.979663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:22.988336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:22.989833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:22.990962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-84cb9e08-7ba4-4982-927b-ad3fe538e10e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.176679 Sep 17 04:29:22.991518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 65/134] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:22 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:23.245556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:23.246165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:23.252722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:23.253321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:23.253761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:23.254456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:23.254906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:23.255317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:23.307094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:23.307854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:23.315043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:23.315983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:23.316544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:23.317359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:23.317910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:23.318444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:23.326275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:23.437067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:23.450198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-59160780-9b2e-473d-bb05-b85ae5901e3c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.207396 Sep 17 04:29:23.454889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 70/135] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:23 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:23.504321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:23.512532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ac9704f8-477c-4836-8d1d-6fbb33adb5f8 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.209749 Sep 17 04:29:23.513139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 66/136] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:23 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:29:24.015938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:24.016678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:24.023039 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.023574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.024010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.024687 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.025268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.025694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.157881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:24.167612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:24.169021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:24.170157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b62bb2b6-d729-4074-8962-9e4184d2370b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.158186 Sep 17 04:29:24.170706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 71/137] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:24 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:24.473834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:24.474417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:24.481072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.481596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.482018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.482656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.483116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.483524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.537681 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:24.538951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:24.545317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.545998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.546435 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.547124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:24.547575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:24.547993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:24.626254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:24.634474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ee83a8e0-81ce-4556-94d4-6023810c7c8c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.164572 Sep 17 04:29:24.635227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 67/138] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:24 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:24.688815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:24.696610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-43ca19ea-401f-4a84-ad07-c55177b648e3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.163317 Sep 17 04:29:24.697359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 72/139] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:24 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:25.198068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:25.199071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:25.209034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.209882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.210598 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.211711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.212453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.213129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.351011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:25.359876 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:25.361463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:25.362521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-88226487-df3c-4314-96c4-d9df42e9cef5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.171147 Sep 17 04:29:25.363079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 68/140] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:25 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:25.660921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:25.661748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:25.671270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.672015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.672944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.674038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.674706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.675125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.719873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:25.720568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:25.727433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.728003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.728431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.729123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:25.729592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:25.730017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:25.808823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:25.816716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-af3112b0-bd24-431a-b75f-71fa4b337071 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.161924 Sep 17 04:29:25.817315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 73/141] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:25 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:25.860069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:25.868044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-44ee505c-9e9f-4dff-8563-869227c7a687 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.152318 Sep 17 04:29:25.868608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 69/142] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:25 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:26.391356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:26.392346 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:26.399423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.399886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.400333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.401046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.401552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.401992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.539096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:26.551191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:26.553539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:26.555196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-cfd13523-f502-41ba-a99f-6f9bc22deb60 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.170189 Sep 17 04:29:26.555947 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 74/143] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:26 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:26.839492 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:26.840093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:26.846581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.847060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.847489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.848169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.848617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.849036 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.890584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:26.891178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:26.897628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.898113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.898548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.899233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:26.899683 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:26.900101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:26.980786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:26.990111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e8e2a881-e904-41e9-b22c-a0a1a96ac394 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1366 microversion: 2.1 time: 0.154502 Sep 17 04:29:26.990895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 70/144] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:26 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:27.031282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:27.038926 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fe912794-334e-4f4f-a2ab-f7b895029b59 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.152210 Sep 17 04:29:27.039492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 75/145] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:26 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:27.586934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:27.587979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:27.598183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:27.598977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:27.599695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:27.600817 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:27.601997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:27.602696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:27.747110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:27.755679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:27.757122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:27.758284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d47164fe-b745-459c-8d79-12b68626465e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.178165 Sep 17 04:29:27.758868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 71/146] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:27 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:28.016420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:28.016976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:28.023465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.023985 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.024583 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.025244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.025705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.026126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.062569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:28.063270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:28.069865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.070471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.070975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.071762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.072387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.072955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.156138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:28.164282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e1b3e671-0b68-4293-abad-a7e29087745b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1556 microversion: 2.1 time: 0.152028 Sep 17 04:29:28.164944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 76/147] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:28 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1556 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:28.187882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:28.188555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:28.195033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.195770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.196458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.197328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.198128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.198697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.211692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:28.222221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d178bac0-2a0c-4c3a-a9e8-a860ec934f25 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.164091 Sep 17 04:29:28.239953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 72/148] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:28 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:28.330739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-681d68f9-16de-43fa-98a7-875a3f173fe4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.146810 Sep 17 04:29:28.331517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 77/149] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:29:28 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-interface => generated 285 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:29:28.783504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:28.784200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:28.790736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.791301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.791764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.792451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:28.792896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:28.793347 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:28.933327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:28.943473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:28.948895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:28.950329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-190df168-5d18-4eea-ae4e-178d55819ab9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.170869 Sep 17 04:29:28.951051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 73/150] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:28 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:29.262323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:29.264657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:29.271131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:29.271896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:29.272530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:29.273505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:29.274159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:29.274804 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:29.605704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:29.614964 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ee3e79c1-8115-4b4c-a309-7a2b8c8e0ee1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.356845 Sep 17 04:29:29.615782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 78/151] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:29 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:29.978016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:29.979293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:29.986203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:29.986751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:29.987391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:29.988066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:29.988659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:29.989104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:30.147353 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:30.158340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:30.160256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:30.162406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-58a7a2b1-5fc2-47c2-81f3-7207a2108088 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.189050 Sep 17 04:29:30.163520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 74/152] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:29 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:30.223399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "508f0590-1d24-42b5-9275-280c6cedff50", "tag": "976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:30.236273 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:30.236709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:30.237171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:30.252704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] Creating event network-changed:976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba for instance 508f0590-1d24-42b5-9275-280c6cedff50 on hv-cinder-85799 Sep 17 04:29:30.258939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-704f692a-a071-4648-a117-e266f8ac948a req-e025f0d0-e991-4cfb-a2f7-197dc1c2a56b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039141 Sep 17 04:29:30.259689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 79/153] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:29:30.314170 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "name": "network-vif-plugged", "status": "completed", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:30.338666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:30.340714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:30.341153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:30.359173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] Creating event network-vif-plugged:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:29:30.365472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-64df7652-d375-42fc-ac63-3e2b44c9d8d8 req-c7ca70bf-025c-437d-92cf-d5cb3b0e1140 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054855 Sep 17 04:29:30.366870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 75/154] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:29:30.639476 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:30.640814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:30.654040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:30.654907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:30.656258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:30.659705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:30.660606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:30.661919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:31.189832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:31.190819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:31.212529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:31.213117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:31.215769 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:31.216807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:31.217551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:31.219227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:31.238419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:31.249423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c22c8b91-2759-4bc8-8bb9-1ec56a0d31b3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1315 microversion: 2.1 time: 0.614720 Sep 17 04:29:31.250126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 80/155] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:30 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1315 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:31.380670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:31.403835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:31.405439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:31.406593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ba634981-c391-4786-a489-cf08737dc96e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.221882 Sep 17 04:29:31.412738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 76/156] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:31 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:32.272949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:32.274198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:32.280011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:32.280670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:32.281354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:32.282772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:32.283272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:32.283692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:32.433262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:32.440062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:32.441281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:32.446037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3cb8ff88-1077-4201-8ce2-95489344febb tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1503 microversion: 2.1 time: 0.178063 Sep 17 04:29:32.447246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 81/157] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:32 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1503 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:32.449219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:32.450716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:32.451147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:32.451863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:32.452284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:32.452711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:32.680729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:32.691597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:32.693731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:32.695097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5b857ece-4a6b-47dc-a911-12c4eb554cfe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1405 microversion: 2.20 time: 0.259553 Sep 17 04:29:32.696185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 77/158] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:32 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1405 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:29:33.721583 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:33.722411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:33.728953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=74 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:33.731459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:33.731970 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:33.732399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:33.733142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:33.733588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:33.733998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:33.913065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:33.927323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:33.931858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:33.934178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2cdffd39-9400-43a8-bc1d-bf664fc4cd29 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1595 microversion: 2.20 time: 0.217169 Sep 17 04:29:33.938064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 82/159] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:29:33 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1595 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:29:33.962497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:33.963004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:33.970028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:33.970899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:33.971404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:33.972659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:33.973695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:33.974268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:34.007923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3989b396-363d-483e-8602-c03aaa1761aa"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:34.011690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:34.017731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:34.018418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:34.019000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:34.019942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:34.020848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:34.021642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:34.140257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5013cd65-2167-4a13-936f-3a0fa0c58093 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.181420 Sep 17 04:29:34.141196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 78/160] 10.222.0.60 () {60 vars in 1316 bytes} [Sat Sep 17 04:29:33 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-interface => generated 284 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:29:34.598177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8769749b-4436-48da-bf2a-f925b152cbc4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "POST /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.594040 Sep 17 04:29:34.598821 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 83/161] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:29:34 2022] POST /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092/os-volume_attachments => generated 194 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:29:36.179691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:36.202338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:36.203064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:36.204917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:36.230834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] Creating event network-changed:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:29:36.241012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-e6851691-5c14-4de3-acc9-3313f15f2dc2 req-4a1a177a-f712-4a10-9a00-73df7f0dd268 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066262 Sep 17 04:29:36.249601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 79/162] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:29:44.185331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:44.186952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:44.195090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:44.198374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:44.199159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:44.199934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:44.200849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:44.201344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:44.202030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:44.229211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:44.235051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:29:44.357819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1dd1ff30-bca4-4f9a-b3bc-f19ad5c8b240 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "DELETE /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 204 len: 0 microversion: 2.1 time: 0.179713 Sep 17 04:29:44.358809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 84/163] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:29:44 2022] DELETE /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 17 04:29:44.368305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:44.370118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:44.378729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:44.379748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:44.380489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:44.381377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:44.382699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:44.383345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:44.568389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:44.584254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-74538f3d-d3db-4054-9d63-a5890de9992a tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.219895 Sep 17 04:29:44.598588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 80/164] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:44 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:45.614798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:45.617045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:45.629781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:45.631481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:45.633208 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:45.636295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:45.638040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:45.639101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:45.901364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:45.911572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-717c5e32-d861-4111-9f06-f27b8aea80f4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.303224 Sep 17 04:29:45.912378 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 85/165] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:45 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:46.963382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:46.964953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:46.988894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:46.989724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:46.990455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:46.991220 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:46.991945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:46.992396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:47.251802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:47.268954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e9cc932-39da-4e43-8db4-97eb5a78258d tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.322571 Sep 17 04:29:47.269683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 81/166] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:46 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:48.293792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:48.294405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:48.307130 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:48.308348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:48.310579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:48.317275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:48.319126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:48.324888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:48.600061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:48.608846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8f84c7c0-c767-44d0-90fd-24cbc7f59ad0 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.320846 Sep 17 04:29:48.624300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 86/167] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:48 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:49.640850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:49.642238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:49.648690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:49.649529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:49.650037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:49.650847 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:49.651367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:49.652802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:49.907435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:49.919438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-75fcbf97-8321-4ba4-8e95-4272c9ea1e03 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.291255 Sep 17 04:29:49.925237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 82/168] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:49 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:50.944717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:50.945384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:50.963277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:50.964068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:50.964859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:50.965614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:50.966215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:50.966785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:51.188213 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:51.205106 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9537eaa6-a752-41f6-9648-0564218aabd3 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.264957 Sep 17 04:29:51.205989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 87/169] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:50 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:52.230430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:52.232103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:52.238203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=46 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:52.240149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:52.241040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:52.242165 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:52.244183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:52.245012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:52.245713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:52.406394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:52.424480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-179a87c2-8899-44ab-a1cb-5710eb18a13e tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.198132 Sep 17 04:29:52.425843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 83/170] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:52 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:53.475005 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:53.477648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:53.497644 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:53.499922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:53.504584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:53.506954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:53.507615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:53.509189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:53.709672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:53.718276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1b9c453d-9e40-4ba3-9202-a8303766246c tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.253305 Sep 17 04:29:53.720774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 88/171] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:53 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:54.742645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:54.744149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:54.750174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:54.757720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:54.758346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:54.759242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:54.759841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:54.760412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:54.764149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=97 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:29:54.959785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:54.968264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9a79d2a2-c8d1-413e-ad17-b3327409ffd4 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1555 microversion: 2.1 time: 0.229472 Sep 17 04:29:54.970193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 84/172] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:54 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1555 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:56.003390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:56.003390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:56.029488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:56.032971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:56.033622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:56.035091 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:56.035543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:56.035981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:56.052915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092", "name": "network-vif-deleted", "tag": "2666bef0-34b1-4359-b664-5517c071bcec"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:29:56.077611 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:56.078425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:56.079076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:56.107884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] Creating event network-vif-deleted:2666bef0-34b1-4359-b664-5517c071bcec for instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 on hv-cinder-85799 Sep 17 04:29:56.134283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-0e0115c7-9b7d-4625-8a2d-ddfa532a1848 req-36bc2fcb-6411-429b-a8f9-b6c4093ce687 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085655 Sep 17 04:29:56.138765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 85/173] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:29:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:29:56.300287 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:56.356450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fc968067-b4d1-4812-bccb-61ac4f635ac7 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1513 microversion: 2.1 time: 0.365078 Sep 17 04:29:56.357453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 89/174] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:55 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1513 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:57.381881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:57.382933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:57.406219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:57.406520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:57.407092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:57.411823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:57.412291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:57.412774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:57.496895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:57.498295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:57.498739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:57.499619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lazy-loading 'fault' on Instance uuid 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:29:57.613524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9be671f062ed4cb793facf9ca7920263', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39cc778e84014ed98f33554b7c46b174', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:29:57.628010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9b030877-0743-4e89-9e57-d0dd00ab6b51 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 200 len: 1351 microversion: 2.1 time: 0.251478 Sep 17 04:29:57.632055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 86/175] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:57 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 1351 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:29:58.653364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:58.654007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:58.660966 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:58.661716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:58.662338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:58.663685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:58.664293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:58.664870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:58.681560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] HTTP exception thrown: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Request method failure captured: Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 HTTP/1.1 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzwPRE0qEwty8Z5iyd5ZbKIGMj5Ewcphy_QrrpjFILkFN3f47c--5eGjGW_0HzDIU8OK9GlzOPcBiqNuoIohwrK0VpLB05DwC8Mf_lrgZf8c1UcBG8o7-WTQ-BPKGCLonzukuShKDqg8UX8EQ425vu8zeeQrILJfSeuVu4g67IPpi0 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:29:58.686152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_39cc778e84014ed98f33554b7c46b174"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 9be671f062ed4cb793facf9ca7920263 Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.687552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092'} Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:29:58.688453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.691196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Returning 404 to user: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:29:58.691196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7fb82d9c-2b50-4e9a-84a4-ee79a53b2f36 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 404 len: 111 microversion: 2.1 time: 0.038201 Sep 17 04:29:58.700195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 90/176] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:29:58 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:29:58.849060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:29:58.852699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:29:58.861593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:58.862985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:58.863646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:58.865045 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:29:58.865791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:29:58.866365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:29:58.888669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] HTTP exception thrown: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Request method failure captured: Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092/os-volume_attachments HTTP/1.1 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzwPRE0qEwty8Z5iyd5ZbKIGMj5Ewcphy_QrrpjFILkFN3f47c--5eGjGW_0HzDIU8OK9GlzOPcBiqNuoIohwrK0VpLB05DwC8Mf_lrgZf8c1UcBG8o7-WTQ-BPKGCLonzukuShKDqg8UX8EQ425vu8zeeQrILJfSeuVu4g67IPpi0 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:29:58.895340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_39cc778e84014ed98f33554b7c46b174"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 9be671f062ed4cb793facf9ca7920263 Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.896577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'server_id': '10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092'} Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:29:58.897327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:29:58.898395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:29:58.899539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Returning 404 to user: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:29:58.899539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dcaf97ab-2157-4b6e-833a-e0913ce1f1c2 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.056428 Sep 17 04:29:58.899539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 87/177] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:29:58 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092/os-volume_attachments => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:30:00.335195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:00.335727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:00.345895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:00.346378 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:00.346806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:00.347739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:00.348185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:00.348839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:00.366472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] HTTP exception thrown: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Request method failure captured: Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 HTTP/1.1 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzwPRE0qEwty8Z5iyd5ZbKIGMj5Ewcphy_QrrpjFILkFN3f47c--5eGjGW_0HzDIU8OK9GlzOPcBiqNuoIohwrK0VpLB05DwC8Mf_lrgZf8c1UcBG8o7-WTQ-BPKGCLonzukuShKDqg8UX8EQ425vu8zeeQrILJfSeuVu4g67IPpi0 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:30:00.368419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_39cc778e84014ed98f33554b7c46b174"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 9be671f062ed4cb793facf9ca7920263 Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.369627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092'} Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:30:00.370345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.371371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.373620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Returning 404 to user: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:30:00.374705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ce25d2b2-e999-4965-b499-8601c4218969 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "DELETE /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 404 len: 111 microversion: 2.1 time: 0.043360 Sep 17 04:30:00.375439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 91/178] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:30:00 2022] DELETE /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:30:00.395388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:00.397275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] [instance: 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:00.404672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:00.405669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:00.406186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:00.407072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:00.407986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:00.408479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:00.425360 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] HTTP exception thrown: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Request method failure captured: Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 HTTP/1.1 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzwPRE0qEwty8Z5iyd5ZbKIGMj5Ewcphy_QrrpjFILkFN3f47c--5eGjGW_0HzDIU8OK9GlzOPcBiqNuoIohwrK0VpLB05DwC8Mf_lrgZf8c1UcBG8o7-WTQ-BPKGCLonzukuShKDqg8UX8EQ425vu8zeeQrILJfSeuVu4g67IPpi0 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:30:00.427720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_39cc778e84014ed98f33554b7c46b174"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/39cc778e84014ed98f33554b7c46b174"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 39cc778e84014ed98f33554b7c46b174 Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-DeleteServersTestJSON-612875130 Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 9be671f062ed4cb793facf9ca7920263 Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-DeleteServersTestJSON-612875130-project Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.429278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092'} Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:30:00.430134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. Sep 17 04:30:00.431487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:30:00.432674 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] Returning 404 to user: Instance 10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:30:00.434956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9fac4401-cb8b-4aca-a158-04ee84a1c8f1 tempest-DeleteServersTestJSON-612875130 tempest-DeleteServersTestJSON-612875130-project] 10.222.0.60 "GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092" status: 404 len: 111 microversion: 2.1 time: 0.053042 Sep 17 04:30:00.445137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 88/179] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:00 2022] GET /compute/v2.1/servers/10ec11a1-cdf0-4adf-b9f5-4c4a0c82e092 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:30:01.712163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0bd1cd74-6722-4481-b093-2dd666a59eac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'create', calling method: .version_select at 0x7fbc4f800280>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1388485820", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPhhZgy8UEi3/rgp1gjf5J8/E2eHdXAG2Xjj/Wm4/1JYaN3TzMjuPCIKhzOdCv48X4rVuMSC4s6vLRxNnbtCGqDWvOA1Y2ueBZOoNnDKISxF+d1iVs+T1A8w2dgEYAvbcA=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:01.734527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-0bd1cd74-6722-4481-b093-2dd666a59eac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Getting quotas for project 4ca16cc178624419954df6bad327f722. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:30:01.740190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-0bd1cd74-6722-4481-b093-2dd666a59eac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Getting quotas for user f6a5aff543234d7a9976c2e43453aef2 and project 4ca16cc178624419954df6bad327f722. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:30:01.781701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0bd1cd74-6722-4481-b093-2dd666a59eac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.073052 Sep 17 04:30:01.782477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 92/180] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:30:01 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:01.920803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1332158073", "imageRef": "883820ca-7ec9-431d-8c60-b41a4d4d80c3", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1388485820", "networks": [{"uuid": "926c6c09-466c-4643-81fe-c6d1e0236acc"}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:01.993275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] validate_networks() for [('926c6c09-466c-4643-81fe-c6d1e0236acc', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:30:02.389483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:30:02.390351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:30:02.391085 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:30:02.391825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:30:02.392794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:30:02.405117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:02.406236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:02.406889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:02.407952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:02.408673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:02.409336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:02.427806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Getting quotas for project 4ca16cc178624419954df6bad327f722. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:30:02.432180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Getting quotas for user f6a5aff543234d7a9976c2e43453aef2 and project 4ca16cc178624419954df6bad327f722. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:30:02.443530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:30:02.445832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='883820ca-7ec9-431d-8c60-b41a4d4d80c3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:30:02.554997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7ea00795-8daf-43f7-b85e-3eb381664812 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.639263 Sep 17 04:30:02.562590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 89/181] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:30:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 648 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:30:02.572421 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1e89c237-45cf-4ee3-928e-d70ff2916835 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:02.574890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1e89c237-45cf-4ee3-928e-d70ff2916835 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:02.675817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1e89c237-45cf-4ee3-928e-d70ff2916835 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:02.684111 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1e89c237-45cf-4ee3-928e-d70ff2916835 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1259 microversion: 2.1 time: 0.120431 Sep 17 04:30:02.685390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 93/182] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:02 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1259 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:03.708943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:03.714524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:03.739236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:03.741897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:03.742528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:03.743966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:03.744887 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:03.745877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:03.936346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:03.952079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f96fce21-6c6b-4648-b03c-f7d083251829 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1255 microversion: 2.1 time: 0.247396 Sep 17 04:30:03.952842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 90/183] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:03 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1255 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:04.977072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:04.977726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:04.996592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:04.997024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:04.997488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:04.998199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:04.998649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:04.999074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:05.124702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:05.142055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-972a304c-1211-48a0-bd95-0050750e2266 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1255 microversion: 2.1 time: 0.169735 Sep 17 04:30:05.143238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 94/184] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:04 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1255 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:06.164358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:06.164971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:06.171187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:06.171893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:06.172325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:06.173259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:06.173719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:06.174288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:06.243061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:06.252589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fac6fb2f-81f9-4355-8bfb-0425472f66ab tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1255 microversion: 2.1 time: 0.091883 Sep 17 04:30:06.253538 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 91/185] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:06 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1255 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:07.276192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:07.276798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:07.284874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:07.285563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:07.285988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:07.286670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:07.287271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:07.287692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:07.354442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:07.362875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2d6c84fb-f52e-4a59-a433-34796e72cb99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1311 microversion: 2.1 time: 0.090865 Sep 17 04:30:07.363440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 95/186] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:07 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1311 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:08.386597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:08.387291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:08.405105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:08.406183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:08.408785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:08.410965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:08.411744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:08.414638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:08.546814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:08.555855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-55d17239-b95e-4066-a528-e6a0961a197e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1319 microversion: 2.1 time: 0.173402 Sep 17 04:30:08.558753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 92/187] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:08 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1319 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:09.586813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:09.588559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:09.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:09.603286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:09.604416 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:09.607443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:09.609747 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:09.612084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:10.676921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:10.697924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9c420d26-85e9-4244-b197-9af4f6177685 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 1.117201 Sep 17 04:30:10.699266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 96/188] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:09 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 1119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:11.723655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:11.728412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:11.745292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:11.750748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:11.758035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:11.760466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:11.761195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:11.767536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:12.204177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:12.214554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f590bbbd-a079-4b02-9490-f1a07eeaac88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.496996 Sep 17 04:30:12.229916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 93/189] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:11 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:12.467913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:12.491002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:12.491350 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:12.497164 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:12.536456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] Creating event network-changed:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:30:12.547244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-64dc55a2-e5b3-4fc7-bb00-c038785b33e5 req-444383b2-56f0-42ed-9ce7-1e17bce6e9e6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083072 Sep 17 04:30:12.550211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 97/190] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:30:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:13.166432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:13.167874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:13.252150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:13.253182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:13.263423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:13.265297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:13.265814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:13.266252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:13.266991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:13.267540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:13.267950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:13.438166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1750509216", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:13.495684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:13.505300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aa5f372c-11cb-4634-8e9e-07bca9b20007 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.257742 Sep 17 04:30:13.506432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 98/191] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:13 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:13.561495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] validate_networks() for [('8d122af4-d997-47f1-a988-a7aa32c2bb1b', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:30:14.088890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:30:14.089578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:30:14.090154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:30:14.090650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:30:14.091146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:30:14.108467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:14.109270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:14.109881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:14.110865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:14.111477 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:14.112077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:14.132412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:30:14.137252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for user 5bb515269d1a45f594f3d297c7fd267e and project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:30:14.150401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:30:14.152765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:30:14.271932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0fecf944-4a15-42bb-894e-a2222cfa0a5e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.114329 Sep 17 04:30:14.279225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 94/192] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:30:13 2022] POST /compute/v2.1/servers => generated 383 bytes in 1124 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:30:14.308661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6e4fcb77-35fa-4358-b0d5-01a54807c484 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:14.311157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6e4fcb77-35fa-4358-b0d5-01a54807c484 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:14.331022 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:14.424486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6e4fcb77-35fa-4358-b0d5-01a54807c484 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:14.437997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6e4fcb77-35fa-4358-b0d5-01a54807c484 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1222 microversion: 2.1 time: 0.153815 Sep 17 04:30:14.454004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 99/193] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:14 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1222 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:14.513142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1619790833", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:14.536932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:14.536932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:14.546916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:14.548900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:14.550776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:14.553021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:14.554406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:14.556352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:14.569761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:14.631189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] validate_networks() for [('8d122af4-d997-47f1-a988-a7aa32c2bb1b', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:30:14.903901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:14.913841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-25e5d19a-93a2-42c2-95a9-d52908db9136 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.385869 Sep 17 04:30:14.917785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 100/194] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:14 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:15.131990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:30:15.132445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:30:15.132922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:30:15.133433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:30:15.133843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:30:15.142856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:15.143345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:15.143773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:15.144492 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:15.145003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:15.145465 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:15.163469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:30:15.167297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for user 5bb515269d1a45f594f3d297c7fd267e and project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:30:15.179206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:30:15.181506 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:30:15.232181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1dda70f6-5cc6-4129-a7ca-748776d820aa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.723184 Sep 17 04:30:15.233401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 95/195] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:30:14 2022] POST /compute/v2.1/servers => generated 383 bytes in 725 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:30:15.245117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8c388ce3-b3c7-4322-85c6-e27e3ac8577a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:15.247330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8c388ce3-b3c7-4322-85c6-e27e3ac8577a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:15.385946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8c388ce3-b3c7-4322-85c6-e27e3ac8577a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:15.397710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8c388ce3-b3c7-4322-85c6-e27e3ac8577a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1222 microversion: 2.1 time: 0.156981 Sep 17 04:30:15.398302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 101/196] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:15 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1222 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:15.941045 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:15.942134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:15.957824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:15.958319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:15.958994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:15.960237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:15.961514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:15.962244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:16.216666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:16.226574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4576b57a-314c-43a3-9fc0-4892ae32f259 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.292790 Sep 17 04:30:16.242284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 96/197] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:15 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:16.423432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:16.424328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:16.431030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:16.432037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:16.432700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:16.433640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:16.434503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:16.435137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:16.502769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:16.511569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e246c51d-ed4e-4463-9aa1-9df6d9986dd7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.092406 Sep 17 04:30:16.526855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 102/198] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:16 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:17.250393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:17.250984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:17.257713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:17.258534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:17.258990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:17.259709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:17.260168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:17.260696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:17.442707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:17.482320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2b05f7d7-9f1e-4ab7-922e-bde7d05a2e1e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.235975 Sep 17 04:30:17.483118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 97/199] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:17 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:17.534350 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:17.534954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:17.545844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:17.546494 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:17.547084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:17.552668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:17.553441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:17.554084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:17.654241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:17.666479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0ed47469-64ce-42c8-99b7-4ea4b5fdf8b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.135077 Sep 17 04:30:17.693309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 103/200] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:17 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:18.514635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:18.515256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:18.522020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:18.522515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:18.522950 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:18.523645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:18.524098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:18.524518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:18.695027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:18.696302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:18.704533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:18.705592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:18.706503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:18.710606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:18.711242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:18.712755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:18.713390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:18.721205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a11fa5c4-0745-4563-a1b5-bd7d9d11cfea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.217109 Sep 17 04:30:18.739192 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 98/201] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:18 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:18.797536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:18.808446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9930992d-0e33-4f8b-b997-d39671d6b610 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.117157 Sep 17 04:30:18.809411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 104/202] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:18 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:19.766959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:19.768069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:19.791211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:19.792094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:19.792630 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:19.793515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:19.794193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:19.795031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:19.835210 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:19.836444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:19.863407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:19.864688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:19.866101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:19.866850 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:19.868880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:19.869348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:19.976485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:19.984921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-16a26c71-8e03-4927-89da-389eef71fedd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.153946 Sep 17 04:30:19.985489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 105/203] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:19 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:20.003565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:20.012247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f0eb550d-390c-4299-a581-9c4f98f1340a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.268394 Sep 17 04:30:20.022893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 99/204] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:19 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:21.008754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:21.010561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:21.019159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:21.020226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:21.020818 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:21.021745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:21.022703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:21.023282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:21.034338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:21.034935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:21.041274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:21.041757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:21.042179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:21.042870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:21.043312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:21.043726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:21.124617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:21.134674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9ac731de-5389-4532-96f3-d2b575488bb3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.130405 Sep 17 04:30:21.135265 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 106/205] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:21 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:21.194203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:21.206207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-615c6732-f545-4a29-a4f2-6f14a1ba9239 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.175721 Sep 17 04:30:21.214494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 100/206] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:21 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:22.163095 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:22.163649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:22.174234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:22.174729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:22.175163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:22.176032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:22.176527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:22.176951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:22.238522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:22.239796 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:22.247857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:22.248419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:22.248908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:22.249532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:22.250229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:22.251617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:22.252386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:22.265801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a0fb3fa6-f810-48a5-ab87-ae85885f4e83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.107484 Sep 17 04:30:22.279411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 107/207] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:22 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:22.433795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:22.453331 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3ee8580c-8348-4a79-b678-bc6dde53fa6c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.220788 Sep 17 04:30:22.455077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 101/208] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:22 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:23.289748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:23.290349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:23.298575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:23.299058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:23.299487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:23.300174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:23.300614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:23.301033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:23.375696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:23.386023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2b6e1855-164f-430b-9758-eb7ea5bfb62e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.100068 Sep 17 04:30:23.387102 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 108/209] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:23 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:23.479149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:23.479807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:23.481668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=9 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:23.487588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:23.488071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:23.488978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:23.489717 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:23.490874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:23.491555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:23.668415 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:23.676854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8a684e2a-75f6-43c6-869a-40eb7d0326ac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.202044 Sep 17 04:30:23.677467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 102/210] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:23 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:24.410445 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:24.411073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:24.421010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:24.421732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:24.422364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:24.423272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:24.423726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:24.424282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:24.490780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:24.499280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-afd50e10-f864-4be6-ad91-b080076dba8a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1226 microversion: 2.1 time: 0.093026 Sep 17 04:30:24.499847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 109/211] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:24 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:24.706788 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:24.707405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:24.715620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:24.718274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:24.719171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:24.720185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:24.720739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:24.721269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:24.916151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:24.918383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:24.929892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-eb90e039-74da-405d-a630-ee9392bcc8ca tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.227071 Sep 17 04:30:24.947199 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 103/212] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:24 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:25.525710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:25.527190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:25.533612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=4 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:25.536330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:25.537857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:25.538372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:25.539301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:25.539942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:25.540570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:25.648892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:25.661864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9feb16a1-8e91-44c4-b6a1-e805dde1349a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1274 microversion: 2.1 time: 0.137662 Sep 17 04:30:25.663021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 110/213] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:25 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1274 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:25.951592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:25.952984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:25.955127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=4 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:25.955773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=12 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:25.961552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:25.962204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:25.962852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:25.964124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:25.964550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:25.965161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:26.103265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:26.112428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6889a74f-2b46-4853-a2f0-286730cc0684 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.164539 Sep 17 04:30:26.113300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 104/214] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:25 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:26.685897 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:26.686431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:26.693425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:26.693845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:26.694282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:26.695017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:26.695464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:26.695987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:26.761357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:26.769741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6aca6cf7-b689-4fea-9f4a-28d032fa6d50 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.087911 Sep 17 04:30:26.770303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 111/215] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:26 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:27.142455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:27.143069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:27.150220 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:27.150757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:27.151232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:27.151991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:27.152490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:27.152962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:27.285046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:27.292893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1ad4de6d-b2c4-41a8-bc03-db11a4e93ee8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.156952 Sep 17 04:30:27.293490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 105/216] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:27 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:27.792813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:27.793446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:27.800248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:27.800741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:27.801204 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:27.801887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:27.802343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:27.802775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:27.863711 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:27.871953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0d661355-4bb7-4fec-888e-f08ba25f040d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.083086 Sep 17 04:30:27.872509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 112/217] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:27 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:28.315903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:28.316480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:28.322986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:28.323465 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:28.323891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:28.324592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:28.325035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:28.325498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:28.455173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:28.463337 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-99a1b504-a8af-4cfd-b700-ba54ce02020a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.151384 Sep 17 04:30:28.463895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 106/218] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:28 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:28.899326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:28.900352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:28.910319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:28.910809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:28.911246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:28.911931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:28.912388 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:28.912811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:28.976163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:28.984018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-73b9ed9b-8de9-463b-bce8-178342bec56d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.091377 Sep 17 04:30:28.984611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 113/219] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:28 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:29.486840 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:29.487415 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:29.493917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:29.494468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:29.494902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:29.495577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:29.496019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:29.496430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:29.624497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:29.632464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-95937ce9-5332-4b24-a4e3-8055537caaea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.149568 Sep 17 04:30:29.633026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 107/220] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:29 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:30.006957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:30.007553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:30.014343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:30.014849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:30.015271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:30.015944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:30.016391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:30.016809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:30.077396 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:30.085516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-47ccbad4-5daa-4db8-bf9a-10b1e53cedaa tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.082362 Sep 17 04:30:30.086072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 114/221] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:30 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:30.655430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:30.656039 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:30.662400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:30.662883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:30.663329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:30.664009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:30.664519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:30.664943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:30.795389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:30.804186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e1207051-95be-46e7-a3f9-93d808de52fc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.152681 Sep 17 04:30:30.804764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 108/222] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:30 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:31.109582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:31.110293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:31.118405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:31.118988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:31.119501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:31.120312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:31.120859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:31.121454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:31.185962 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:31.194012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a393ea5d-8ecb-4976-8513-40ce47228e89 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.089043 Sep 17 04:30:31.194579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 115/223] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:31 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:31.827885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:31.828599 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:31.835386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:31.836085 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:31.836873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:31.837594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:31.838036 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:31.838451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:31.979853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:31.988418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d1242c3f-23da-46f1-9e71-26a5c5fe53dc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.164937 Sep 17 04:30:31.989110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 109/224] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:31 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:32.216164 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:32.216672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:32.223652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:32.224138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:32.224576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:32.225690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:32.226139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:32.226529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:32.290888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:32.298872 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7def6ea7-88f5-4726-a893-f526ecdafa14 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.086744 Sep 17 04:30:32.299466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 116/225] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:32 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:33.011498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:33.012103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:33.018466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:33.019052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:33.019486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:33.020176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:33.020630 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:33.021051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:33.173498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:33.182508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6e887a1c-4d9e-4a91-b468-8f5e3335844a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.175067 Sep 17 04:30:33.183243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 110/226] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:33 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:33.321127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:33.322286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:33.329744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:33.330288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:33.330785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:33.331520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:33.331958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:33.333062 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:33.417312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:33.418520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:33.428715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f8f8f7be-25a4-417f-8167-e109c1fbcab2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.111192 Sep 17 04:30:33.429757 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 117/227] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:33 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:34.206250 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:34.206954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:34.213459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:34.213933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:34.214381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:34.215066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:34.215513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:34.215975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:34.352948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:34.360930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-90751e64-636c-4143-b37b-1ac3bce70c33 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.158566 Sep 17 04:30:34.361560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 111/228] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:34 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:34.451122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:34.452332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:34.458958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:34.459447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:34.460073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:34.460727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:34.461271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:34.461706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:34.526407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:34.534480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bfdbfcad-9b7a-48ee-a715-68f6ddf96e1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.087160 Sep 17 04:30:34.535087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 118/229] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:34 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:35.387450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:35.388948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:35.396985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:35.397551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:35.397986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:35.398680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:35.399132 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:35.399551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:35.558004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:35.558621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:35.566444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:35.569707 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:35.570193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:35.570686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:35.571307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:35.571922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:35.572485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:35.575178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-114d486b-5208-4cb4-b5f2-71868a37887b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.195206 Sep 17 04:30:35.576093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 112/230] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:35 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:35.651387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:35.663948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5a4a7c97-1a8a-4028-bc4c-0240b3eb589d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1282 microversion: 2.1 time: 0.111100 Sep 17 04:30:35.664535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 119/231] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:35 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1282 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:36.602375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:36.603928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:36.618775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:36.620254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:36.621500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:36.624226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:36.625974 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:36.627500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:36.689841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:36.690529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:36.710728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:36.711227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:36.711661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:36.712505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:36.712858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:36.718434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:36.954808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:36.963192 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:36.968551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-820d9158-4893-481c-bb21-2a023ef3acde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1280 microversion: 2.1 time: 0.284548 Sep 17 04:30:36.969889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 120/232] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:36 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1280 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:36.986867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0fe62349-cf48-4e26-a058-36d937f82f99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.391268 Sep 17 04:30:36.989960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 113/233] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:36 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:37.913649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c0837134-994e-4402-a813-3bacee6f4853", "tag": "55d9eabb-ceb8-459c-a4ad-0c9da0df432e"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:37.942649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:37.945250 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:37.946040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:37.976048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] Creating event network-changed:55d9eabb-ceb8-459c-a4ad-0c9da0df432e for instance c0837134-994e-4402-a813-3bacee6f4853 on hv-cinder-85799 Sep 17 04:30:37.989357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-c339263d-fcc1-4a63-a472-3c703e5ed290 req-4e72d6e8-2d4d-4852-a1e9-065d22c884ad service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079179 Sep 17 04:30:37.991609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 121/234] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:30:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:37.995485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:37.996082 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:38.006493 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:38.006991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:38.007427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:38.008131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:38.008582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:38.019878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:38.032304 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:38.034517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:38.054764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:38.060528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:38.061311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:38.063368 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:38.065376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:38.068401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:38.164147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:38.186698 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9b847636-6b64-4f6b-a4c2-7eb033a2763b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1280 microversion: 2.1 time: 0.195239 Sep 17 04:30:38.187277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 114/235] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:37 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1280 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:38.248783 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "name": "network-vif-plugged", "status": "completed", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:38.270236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:38.271342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:38.271984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:38.297676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] Creating event network-vif-plugged:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:30:38.309237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-289f9858-63f7-49f1-a779-e82e369a7ddf req-a2d3d402-820b-4334-a657-b66a64b7bbd5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060555 Sep 17 04:30:38.310357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 115/236] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:39.069927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:39.088590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-90411e90-e6bb-4e46-9de1-b606cb6e9d79 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 1.067689 Sep 17 04:30:39.089220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 122/237] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:38 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 1070 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:39.220316 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:39.220913 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:39.240420 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:39.240942 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:39.241410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:39.243321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:39.247296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:39.247296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:39.476807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:39.486472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-153d0bd7-8c13-4cf8-abc8-91399cbf256a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.275491 Sep 17 04:30:39.501643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 116/238] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:39 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:40.115471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:40.116115 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:40.124131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:40.124617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:40.125048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:40.125770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:40.126220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:40.126637 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:40.273897 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:40.285687 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7c14b3ea-ba58-468d-8cd8-73222f59ccc6 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.174968 Sep 17 04:30:40.286272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 123/239] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:40 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:40.509947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:40.510639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:40.517234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:40.517883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:40.518318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:40.519300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:40.519757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:40.520344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:40.671214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:40.683563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0406f88b-b822-4b6b-9563-9231d7736951 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.177462 Sep 17 04:30:40.685235 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 117/240] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:40 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:41.308617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:41.310116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:41.318101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:41.318588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:41.319216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:41.320104 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:41.320935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:41.321443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:41.463942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:41.472348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-79227df8-be15-47c0-89a5-69f91b4888e7 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.168342 Sep 17 04:30:41.486515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 124/241] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:41 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:41.710891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:41.711511 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:41.718560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:41.718560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:41.719327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:41.720816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:41.721329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:41.721851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:41.922997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:41.932961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-be7dc066-9db6-4384-bb86-1f14f6eb03f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.229785 Sep 17 04:30:41.934112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 118/242] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:41 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:42.498524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:42.499126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:42.505923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:42.506415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:42.506838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:42.507536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:42.507985 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:42.508405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:42.876865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:42.885219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c954b1a4-651c-4884-96d2-cd5f8c9aaecf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.390574 Sep 17 04:30:42.885920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 125/243] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:42 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:42.958697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:42.959866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:42.966243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:42.966731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:42.967160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:42.967849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:42.968897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:42.969355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:43.385408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:43.395162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2f722a45-72e8-4702-a35b-0e9f49e4f566 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.440412 Sep 17 04:30:43.396597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 119/244] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:42 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:43.907978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:43.908584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:43.920194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:43.922198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:43.922794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:43.923874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:43.926389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:43.926979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:44.419615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:44.421787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:44.422384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:44.435835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-406ff87f-5524-40b1-b5a2-2a2ff7441525 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.528804 Sep 17 04:30:44.436771 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:44.437682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:44.438082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 126/245] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:43 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:44.438596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:44.440593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:44.441692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:44.442369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:44.593681 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "33c2f844-d503-44e7-86dd-b67034f5ca40", "tag": "4de14b41-d3ca-42f2-b3cd-1e608f2d4c84"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:44.606274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=60 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:44.608985 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:44.609751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:44.610979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:44.626405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] Creating event network-changed:4de14b41-d3ca-42f2-b3cd-1e608f2d4c84 for instance 33c2f844-d503-44e7-86dd-b67034f5ca40 on hv-cinder-85799 Sep 17 04:30:44.634552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-b3aac100-ec77-4e54-a568-569d4b8fdd3f req-8e24d204-b33f-4e4d-966f-c3f266f51f98 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043373 Sep 17 04:30:44.635946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 127/246] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:30:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:30:44.674894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:44.682686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5a8ed368-5b06-4c70-a202-e38897889d20 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.264812 Sep 17 04:30:44.683265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 120/247] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:44 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:45.476457 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:45.477480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:45.488171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:45.488794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:45.489401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:45.490486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:45.492758 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:45.493433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:45.678351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:45.689531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e04c1206-7406-401d-9b3e-30522b58f0e8 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.233269 Sep 17 04:30:45.690242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 128/248] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:45 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:45.707262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:45.708077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:45.715570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:45.716986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:45.717588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:45.719185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:45.719727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:45.720222 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:45.725526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:45.942056 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:45.960704 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-10fc907c-8043-4b13-ac80-41008f7e2768 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.259246 Sep 17 04:30:45.973836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 121/249] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:45 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:46.713161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:46.714440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:46.721254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:46.721998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:46.722426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:46.723107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:46.728768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:46.729245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:46.957702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:46.969072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-94dbf96d-c814-4129-9ef1-128a16ec601b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.259802 Sep 17 04:30:46.982315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 129/250] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:46 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:46.986927 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:46.987790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:46.996119 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:46.996654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:46.997174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:47.001735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:47.002232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:47.002744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:47.195912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:47.204939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-661c5dfe-9fc1-4f27-8777-6ec0c193dd8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.222003 Sep 17 04:30:47.213731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 122/251] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:46 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:47.995669 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:47.996284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:48.005405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:48.008723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:48.009539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:48.010741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:48.011426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:48.012030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:48.230581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:48.231453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:48.240421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:48.241822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:48.244354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:48.245355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:48.246129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:48.246751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:48.324649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:48.347421 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-825cbe20-eca6-484f-92de-24e7c52b9fc3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1359 microversion: 2.1 time: 0.356212 Sep 17 04:30:48.348587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 130/252] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:47 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1359 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:48.747812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:48.756264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-38e09313-d0a0-499a-b089-631f793b3633 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.529407 Sep 17 04:30:48.762618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 123/253] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:48 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:49.374564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:49.375774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:49.391059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.391500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.393623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.395352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.396549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.396987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.619291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:49.629191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b6557361-a784-440b-9fba-702819a0b21a tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1548 microversion: 2.1 time: 0.262360 Sep 17 04:30:49.629780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 131/254] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:49 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1548 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:49.653070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:49.653779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:49.674604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.675334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.675895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.676749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.677392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.677966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.779153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:49.780065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:49.787135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.787858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.788446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.789315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:49.790017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:49.790579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:49.967868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:49.976620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-11dc6469-1068-4c61-a3fd-f1274e0feadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.201251 Sep 17 04:30:49.977440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 132/255] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:49 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:50.219777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:50.228743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-76590f0a-808f-4318-83a0-3b9f797a30bf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1548 microversion: 2.1 time: 0.580245 Sep 17 04:30:50.229473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 124/256] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:49 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1548 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:50.250897 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:50.255965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:50.256725 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Searching by: {'deleted': False, 'project_id': '4ca16cc178624419954df6bad327f722'} {{(pid=120626) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 17 04:30:50.263780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:50.264254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:50.264859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:50.265622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:50.266064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:50.266654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:50.379320 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.multi_cell_list [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120626) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 17 04:30:50.506529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-eadf00f3-e89e-4c64-8c1c-99fa2de6cc08 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.259468 Sep 17 04:30:50.507392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 133/257] 10.222.0.60 () {58 vars in 1125 bytes} [Sat Sep 17 04:30:50 2022] GET /compute/v2.1/servers => generated 343 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:50.522010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:50.522696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:50.534409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:50.534957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:50.535388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:50.536093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:50.536543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:50.536962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:50.750093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:50.763720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1f91d310-6a1e-48c9-bc13-908c60de1457 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1548 microversion: 2.1 time: 0.247716 Sep 17 04:30:50.764957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 125/258] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:50 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1548 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:51.002831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:51.004382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:51.013334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:51.013819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:51.014433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:51.015128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:51.015572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:51.015984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:51.081646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-47d7b6dc-3445-4d48-b427-9ac8fe0b124b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Action: 'create', calling method: .version_select at 0x7fbc4f8fcb80>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-623666681", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAEaMqsX1N0rcvOso7BYqnoHOQikeuQixA/mNCoGM0Kw4s5aNNFhzRPP2vW4c8cYjw1O32bpdFiXEcBROFAotclkQL5qyra+GO1vg8vG19dHp7XncA2np2CDSaOTDb8Hcg=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:51.102738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-47d7b6dc-3445-4d48-b427-9ac8fe0b124b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:30:51.107640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-47d7b6dc-3445-4d48-b427-9ac8fe0b124b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for user ff241cbd3f7c47328ac5ef2ed4c66e18 and project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:30:51.156251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-47d7b6dc-3445-4d48-b427-9ac8fe0b124b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.314046 Sep 17 04:30:51.158776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 134/259] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:30:50 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:30:51.291689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:51.304736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cdf1882c-4e91-4f9f-ab90-7f406a3b2e16 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.306608 Sep 17 04:30:51.306131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 126/260] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:50 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:52.333172 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:52.333799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:52.340037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:52.340806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:52.341373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:52.342268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:52.342723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:52.343141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:52.774496 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:52.782257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a914ba35-8bb8-434f-a9b8-bbb591a552f1 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.454094 Sep 17 04:30:52.796108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 135/261] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:52 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:53.811454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:53.813814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:53.818150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=50 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:53.829278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:53.829712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:53.830104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:53.831091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:53.831294 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:53.831736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:54.094590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:54.116071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4fc5c469-6437-4b43-8d08-93fc0fbde142 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.312351 Sep 17 04:30:54.116662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 127/262] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:53 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:54.887130 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0ba75f9b-6b57-4c6f-9290-3319432abf32"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:54.889902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:54.896842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:54.897664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:54.898275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:54.899531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:54.900160 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:54.900816 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:54.920632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:30:55.144838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:55.145470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:55.153196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:55.153677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:55.154104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:55.154792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:55.155231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:55.155672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:55.304012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:55.321430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5a33974f-a14b-4e46-8b02-13c9f5f9bd76 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.181894 Sep 17 04:30:55.338847 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 128/263] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:55 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:30:56.101295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c0837134-994e-4402-a813-3bacee6f4853", "name": "network-vif-plugged", "status": "completed", "tag": "55d9eabb-ceb8-459c-a4ad-0c9da0df432e"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:30:56.113760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:56.114567 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:56.115345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:56.127075 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] Creating event network-vif-plugged:55d9eabb-ceb8-459c-a4ad-0c9da0df432e for instance c0837134-994e-4402-a813-3bacee6f4853 on hv-cinder-85799 Sep 17 04:30:56.132309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-d4a1f5df-14c5-489a-9919-f81be4ac62ef req-9cb0deef-556f-42d6-8357-d1bfc5bed10b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036253 Sep 17 04:30:56.133750 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 129/264] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:30:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:30:56.346715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:56.347620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:56.354561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:56.355261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:56.355759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:56.356460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:56.356913 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:56.357497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:56.495520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:56.503601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-87ee00c1-f76b-441b-83ea-f41555d002ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.161587 Sep 17 04:30:56.504176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 130/265] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:56 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:57.531799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:57.531799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:57.535019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:57.535521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:57.535958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:57.536659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:57.537131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:57.537897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:57.684146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:57.692935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4b73b29b-1b95-48f3-8e74-1229c8cdab94 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.169501 Sep 17 04:30:57.693554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 131/266] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:57 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:58.716012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:58.716615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:58.723081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:58.723581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:58.724005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:58.725053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:58.725543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:58.725967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:58.859244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:30:58.867163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6b1933d7-db75-4863-843e-44e42e00d8f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.155082 Sep 17 04:30:58.867705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 132/267] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:58 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:30:59.892298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:30:59.893177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:30:59.900050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:59.900535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:59.900968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:30:59.901693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:30:59.902143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:30:59.902562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:00.042973 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:00.051588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-19842c2c-a071-4379-88d6-4330876086ce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.164677 Sep 17 04:31:00.052156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 133/268] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:30:59 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:00.298712 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:01.074210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:01.074817 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:01.080978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:01.081512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:01.081916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:01.082628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:01.083071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:01.083487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:01.115697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=4 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:01.211149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:01.219091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5f0c0438-2598-4d94-bf81-96e4e7b780d6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.148905 Sep 17 04:31:01.219665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 134/269] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:01 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:02.241987 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:02.242697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:02.248675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:02.249168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:02.249654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:02.250342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:02.250795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:02.251219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:02.415979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:02.428206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-104d4e35-dc68-4c70-b994-b8954afedeeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.190136 Sep 17 04:31:02.429583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 135/270] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:02 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:02.536654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-add89835-128e-4315-83ed-abbe101cccb9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 7.653586 Sep 17 04:31:02.537298 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 136/271] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:30:54 2022] POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments => generated 194 bytes in 7655 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:03.457725 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:03.459099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:03.465920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:03.466521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:03.467085 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:03.467780 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:03.468236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:03.468662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:03.627572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:03.637786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-aa1c438b-a163-4fce-9fc2-7e8b8b933b7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.186284 Sep 17 04:31:03.638360 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 136/272] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:03 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:04.452749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "33c2f844-d503-44e7-86dd-b67034f5ca40", "name": "network-vif-plugged", "status": "completed", "tag": "4de14b41-d3ca-42f2-b3cd-1e608f2d4c84"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:04.464864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:04.465364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:04.465865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:04.477833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] Creating event network-vif-plugged:4de14b41-d3ca-42f2-b3cd-1e608f2d4c84 for instance 33c2f844-d503-44e7-86dd-b67034f5ca40 on hv-cinder-85799 Sep 17 04:31:04.487902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-d7b87a52-600a-4142-9531-52c4ba69f8c4 req-0292e549-0255-4047-85a2-99fd32f67559 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040931 Sep 17 04:31:04.488465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 137/273] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:31:04.663054 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:04.663962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:04.669827 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:04.670026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:04.670686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:04.671899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:04.672547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:04.673211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:04.840245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:04.848426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7571216c-0153-4921-9a05-265248da302d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.189533 Sep 17 04:31:04.849581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 137/274] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:04 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:05.873876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:05.875874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:05.881407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:05.881899 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:05.882333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:05.883038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:05.883491 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:05.883952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:06.039834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:06.051392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-40744a8a-fc7f-4be4-91c9-cd5b418cdc78 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1322 microversion: 2.1 time: 0.181869 Sep 17 04:31:06.053037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 138/275] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:05 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1322 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:07.074100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:07.074777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:07.081037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.081563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.081996 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.082690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.083176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.085745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.102637 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:07.250813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:07.260888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-909f8f06-ace5-46a8-81d3-943c84a54130 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1516 microversion: 2.1 time: 0.190612 Sep 17 04:31:07.261522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 138/276] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:07 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1516 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:07.284915 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:07.287623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:07.294332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.294814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.295260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.295940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.296406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.296843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.322602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'flavor' on Instance uuid 33c2f844-d503-44e7-86dd-b67034f5ca40 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:31:07.429300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a7ff8817-29f3-4371-bb64-3cdb6a337b7f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40/action" status: 200 len: 32 microversion: 2.1 time: 0.149192 Sep 17 04:31:07.429912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 139/277] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:31:07 2022] POST /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40/action => generated 32 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:31:07.439385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:07.439986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:07.446818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.447261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.447695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.448375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:07.448831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:07.449291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:07.594370 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:07.602916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4ad925e4-00c3-4abf-a5ef-569e47f404d5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.167318 Sep 17 04:31:07.603483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 139/278] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:07 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:08.629825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:08.631352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:08.640773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:08.641342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:08.641837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:08.642543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:08.643027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:08.643450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:08.844093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:08.852838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1b37f871-030e-48b0-9e56-a3360dc17c7b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.229224 Sep 17 04:31:08.853593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 140/279] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:08 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:09.875795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:09.876409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:09.882800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:09.883284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:09.883726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:09.884411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:09.884861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:09.885311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:10.065855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:10.069792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-983636cf-2355-44fe-af34-d203a2664a1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.197865 Sep 17 04:31:10.070661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 140/280] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:09 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:11.092057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:11.092957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:11.098968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:11.099794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:11.100218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:11.100935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:11.101414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:11.101837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:11.267458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:11.275402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8117e965-8b50-4473-996d-3483fa37101d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.187327 Sep 17 04:31:11.275977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 141/281] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:11 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:12.299954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:12.300697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:12.315775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:12.315775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:12.316387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:12.317291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:12.318094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:12.318249 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:12.490633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:12.491294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:12.498241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:12.498757 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:12.499174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:12.499875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:12.500451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:12.500874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:12.525031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:12.534385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-09c5aeb6-5e57-4284-8081-e47577af1a83 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.238854 Sep 17 04:31:12.534961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 141/282] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:12 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:12.706798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:12.715145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c7f6f1a1-cb8d-4933-a2e1-c2c7216a77ce tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1594 microversion: 2.1 time: 0.228532 Sep 17 04:31:12.729771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 142/283] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:12 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1594 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:12.754986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1350029737", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:13.560425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:13.561170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:13.567839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:13.568446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:13.568939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:13.569710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:13.570220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:13.570708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:13.708047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:13.715683 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-748cb318-e75f-4ee2-b111-1f5402987c9c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.159855 Sep 17 04:31:13.716248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 143/284] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:13 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:14.744690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:14.745417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:14.747873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:14.754309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:14.754885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:14.755292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:14.755995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:14.756434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:14.756849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:14.963038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:14.977118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-58650495-8e67-4ee1-9778-e94bcf79f791 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.236189 Sep 17 04:31:14.977900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 144/285] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:14 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:14.994735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:15.005889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:15.006708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:15.007386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:15.018642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] Creating event network-changed:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:31:15.024305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-48ab0939-98e4-43c2-a332-02bf0041bc99 req-6aac8b6d-cc1c-459c-b5ad-f9271ac20064 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044810 Sep 17 04:31:15.024988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 145/286] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:15.957843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1481957129"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:15.958742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:15.966231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:15.966755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:15.967227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:15.968016 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:15.968641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:15.969070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.111377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.network.security_group_api [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Adding security group 214549b0-5815-46e0-8206-a4e7a53a3e84 to port d54c1156-b70c-4805-bad8-7b71a108b5c0 Sep 17 04:31:16.441715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2ee90df8-a70d-469e-adc0-1ecc28fe29fa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action" status: 202 len: 0 microversion: 2.1 time: 0.489943 Sep 17 04:31:16.446475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 146/287] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:31:15 2022] POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action => generated 0 bytes in 495 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:31:16.451532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:16.452332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:16.459987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:16.460443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:16.461179 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.461900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:16.462482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:16.463141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.600950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:16.609190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-514ef92d-7aa1-4d06-96e9-fadb0d7b668a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.161478 Sep 17 04:31:16.609772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 147/288] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:16 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:16.614416 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:16.615009 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:16.621351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:16.621872 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:16.622349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.623023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:16.623501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:16.623924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.794338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:16.803750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f86dcd80-eefb-4bd9-84cc-cdfaa87f6b8e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1641 microversion: 2.1 time: 0.192734 Sep 17 04:31:16.809712 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 148/289] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:16 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1641 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:16.809915 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:16.821213 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:16.821760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:16.822271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:16.833751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] Creating event network-changed:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:31:16.838663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-bd877e1d-268a-4148-80d6-310a4f97c3d2 req-f27da1ec-23ed-4c1b-8e8e-6a7772fbb387 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032640 Sep 17 04:31:16.839242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 149/290] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:17.634271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:17.635181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:17.641772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:17.642279 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:17.642696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:17.643389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:17.643828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:17.644247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:17.782716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:17.790568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-93367337-dfb8-4e1c-938e-375e4a68fc37 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.161625 Sep 17 04:31:17.791144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 150/291] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:17 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:18.814090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:18.814799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:18.823957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:18.824475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:18.824902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:18.825648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:18.826101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:18.826522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:18.964987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:18.972769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2b1b2cc5-8c4a-4ff3-a938-49e7c45d9478 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.162849 Sep 17 04:31:18.973444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 151/292] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:18 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:19.012955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] validate_networks() for None {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:31:19.294222 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:31:19.294639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:31:19.295111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:31:19.295514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:31:19.295922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:31:19.308040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:19.308751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:19.309450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:19.311268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:19.311848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:19.312410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:19.337138 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Getting quotas for project 6619450eef5a4a5b8d3b6a45fe412788. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:31:19.341579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Getting quotas for user 5817b0d9e6904be2b0ea482746d3d70f and project 6619450eef5a4a5b8d3b6a45fe412788. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:31:19.353487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:31:19.355786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:31:19.405203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9df3429c-92b2-45ea-8acc-7eb8351aa1d7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 6.659397 Sep 17 04:31:19.405922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 142/293] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:31:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 6661 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:31:19.417402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6dc1ff06-9dbf-48ab-9aa3-ea4a38042b9d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:19.418031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6dc1ff06-9dbf-48ab-9aa3-ea4a38042b9d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:19.472384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6dc1ff06-9dbf-48ab-9aa3-ea4a38042b9d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:19.480706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6dc1ff06-9dbf-48ab-9aa3-ea4a38042b9d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1215 microversion: 2.1 time: 0.067155 Sep 17 04:31:19.481281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 152/294] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:19 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1215 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:20.000301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:20.001285 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:20.011535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:20.012332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:20.013041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:20.014214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:20.014972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:20.015669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:20.168478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:20.176933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-741883d9-d7a2-4600-b6a8-91a2c53ece46 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.183503 Sep 17 04:31:20.177539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 143/295] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:19 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:20.506533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:20.507198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:20.516769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:20.517339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:20.517815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:20.518561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:20.519059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:20.519530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:20.522543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:20.583672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:20.591142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b16a31b9-5819-404c-a06f-3ce8d98c8d8d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1219 microversion: 2.1 time: 0.089789 Sep 17 04:31:20.591973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 153/296] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:20 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:21.204276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:21.205139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:21.215213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:21.216016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:21.216736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:21.217900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:21.218677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:21.219336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:21.357735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:21.369310 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4ca8ab0d-e67c-4dac-b24f-9fd3650d956d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.171349 Sep 17 04:31:21.370041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 144/297] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:21 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:21.617810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:21.618473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:21.626221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:21.626743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:21.627216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:21.627972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:21.628470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:21.628934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:21.695017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:21.702782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-80dc84b3-c956-4215-830a-a32aa9f4d1db tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1219 microversion: 2.1 time: 0.090067 Sep 17 04:31:21.703400 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 154/298] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:21 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:22.399884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:22.400834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:22.411468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:22.412263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:22.412965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:22.414140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:22.414878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:22.415567 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:22.552911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:22.561355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e3babf39-87d2-4608-a52f-e831ea2e0431 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.168162 Sep 17 04:31:22.561902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 145/299] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:22 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:22.726174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:22.726773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:22.733607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:22.734120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:22.734532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:22.735243 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:22.735696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:22.736469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:22.803631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:22.812078 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5e574239-a9fe-4052-bbdf-fa7e48f89509 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1219 microversion: 2.1 time: 0.090173 Sep 17 04:31:22.812622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 155/300] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:22 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:23.584560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:23.585176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:23.591262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:23.591670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:23.592089 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:23.592941 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:23.593425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:23.593841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:23.726801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:23.734961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d7e32422-962a-406f-80e3-2174a5df45c2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.154418 Sep 17 04:31:23.735519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 146/301] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:23 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:23.835356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:23.836050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:23.842955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:23.843424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:23.843969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:23.844897 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:23.845385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:23.845870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:23.936922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:23.945061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-681bc1c7-0065-4ad4-a626-bb51850d265c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1219 microversion: 2.1 time: 0.113763 Sep 17 04:31:23.945654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 156/302] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:23 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1219 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:24.758746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:24.759454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:24.761474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=51,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:24.776609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:24.777262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:24.789833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:24.790742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:24.791300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:24.791815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:24.968559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:24.969799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:24.983882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:24.984385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:24.984824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:24.985550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:24.986012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:24.986526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:25.044075 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:25.052150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-59d4f340-6c43-4559-8a0c-c38f879c0781 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.297850 Sep 17 04:31:25.052724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 147/303] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:24 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:25.058983 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:25.066907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-da073b52-11e3-49ec-a462-37470dd3931c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1267 microversion: 2.1 time: 0.103009 Sep 17 04:31:25.067555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 157/304] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:24 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1267 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:26.075525 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:26.076154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:26.082354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:26.082869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:26.083309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:26.083991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:26.084440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:26.084864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:26.090460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:26.091063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:26.097677 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:26.098156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:26.098580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:26.099625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:26.100063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:26.100479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:26.161492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:26.169534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f2425480-3bd0-4244-a9b1-0181036487c3 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1275 microversion: 2.1 time: 0.083097 Sep 17 04:31:26.170078 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 158/305] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:26 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1275 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:26.219328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:26.227742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-132cefc8-d427-4116-b3d5-01174933e500 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.156097 Sep 17 04:31:26.228301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 148/306] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:26 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:27.196462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:27.197064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:27.204334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:27.204853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:27.205353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:27.206079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:27.206563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:27.207011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:27.250902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:27.251516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:27.257987 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:27.258472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:27.259354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:27.260526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:27.261043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:27.261431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:27.278604 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:27.286837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-be7885a2-76e6-451e-9cdb-ca0ef4d818e6 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1275 microversion: 2.1 time: 0.096839 Sep 17 04:31:27.287423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 159/307] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:27 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1275 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:27.397661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:27.405683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-be062a56-bca7-4e16-9f36-04367a6a29b7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.158786 Sep 17 04:31:27.406252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 149/308] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:27 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:28.314232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:28.315285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:28.326389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:28.327092 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:28.327809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:28.328944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:28.329779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:28.330477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:28.399247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:28.407227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7da45846-98ca-445b-8321-32d4ef285dc1 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1275 microversion: 2.1 time: 0.099663 Sep 17 04:31:28.407788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 160/309] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:28 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1275 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:28.429014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:28.429646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:28.436299 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:28.436780 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:28.437253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:28.437953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:28.438409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:28.438824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:28.590929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:28.599393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4d94c422-a9b8-443b-96a1-a631cc03fe9d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.173933 Sep 17 04:31:28.599920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 150/310] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:28 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:29.431717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:29.432370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:29.438762 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:29.439239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:29.439670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:29.440385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:29.440831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:29.441308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:29.477786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:29.502788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:29.510932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-de675db0-719b-4325-befa-0723acd6f4b5 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1275 microversion: 2.1 time: 0.083169 Sep 17 04:31:29.511498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 161/311] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:29 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1275 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:29.627290 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:29.628324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:29.631234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:29.632315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:29.642783 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:29.643582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:29.644313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:29.645500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:29.646261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:29.646960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:29.777829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:29.785979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5c01faa0-e8ec-4f1d-af26-f0699bf62b73 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.165318 Sep 17 04:31:29.786549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 151/312] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:29 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:30.535366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:30.536391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:30.543536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:30.544258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:30.544706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:30.545520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:30.546011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:30.546776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:30.760367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:30.769702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0b8554d4-21f2-4f3d-84c7-6f9d75115537 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1273 microversion: 2.1 time: 0.239503 Sep 17 04:31:30.785731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 162/313] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:30 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1273 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:30.808462 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:30.809653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:30.815606 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:30.816336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:30.816757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:30.817471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:30.817896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:30.818489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:30.976057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:30.985286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-770b95a6-ad48-46ae-9f36-66470df7afd3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.181092 Sep 17 04:31:30.985914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 152/314] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:30 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:31.794089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:31.794089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:31.801229 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:31.801722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:31.802179 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:31.802864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:31.803331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:31.803767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:32.008181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:32.008862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:32.015179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:32.015702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:32.016117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:32.016844 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:32.017345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:32.017837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:32.055253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:32.063575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d87cbc9f-8740-4fdb-81e2-cad7532f3d79 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.274636 Sep 17 04:31:32.064384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 163/315] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:31 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:32.149561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:32.157329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2396680d-ec14-4e42-b5d2-5931bc9def64 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.153160 Sep 17 04:31:32.157899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 153/316] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:32 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:32.647233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5198b50b-0860-472e-b24d-bb4e17962392", "tag": "3716a257-cc8a-4082-8240-219d7f1c5cb6"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:32.665755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:32.666620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:32.667461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:32.680770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] Creating event network-changed:3716a257-cc8a-4082-8240-219d7f1c5cb6 for instance 5198b50b-0860-472e-b24d-bb4e17962392 on hv-cinder-85799 Sep 17 04:31:32.686095 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-93993d71-cfaf-4561-8fcd-823789d560f9 req-fb673b92-34e0-4686-b8e8-0ae8fa8f9c2a service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043859 Sep 17 04:31:32.686663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 164/317] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:31:33.086563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:33.087241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:33.093459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:33.094144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:33.094567 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:33.095269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:33.095888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:33.096303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:33.181981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:33.182676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:33.189336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:33.189828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:33.190280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:33.190953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:33.191407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:33.191830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:33.246784 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:33.254580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5361590f-b2d9-425c-abf9-7e1576e02073 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.171948 Sep 17 04:31:33.255184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 154/318] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:33 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:33.322034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:33.330073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dd7c0cf4-9d97-4402-9d46-1986f5f7a923 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.152714 Sep 17 04:31:33.330668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 165/319] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:33 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:34.282083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:34.283024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:34.293327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:34.294111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:34.294818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:34.295925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:34.296706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:34.297457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:34.353585 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:34.354301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:34.360684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:34.361202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:34.361639 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:34.362326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:34.362780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:34.363238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:34.433817 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:34.435996 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:34.442660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a6786203-da82-4950-a22c-d73d211cfe0d tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.167121 Sep 17 04:31:34.443386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 155/320] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:34 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:34.497290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:34.505068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1c79259c-89cf-4212-9619-33d493a9b4b8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.156004 Sep 17 04:31:34.505650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 166/321] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:34 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:35.470237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:35.471235 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:35.480428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:35.480911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:35.481382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:35.482068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:35.482523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:35.482946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:35.528034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:35.528817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:35.536948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:35.537517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:35.537986 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:35.538821 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:35.539319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:35.539743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:35.620529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:35.629308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bb7d437a-1b19-4fac-8e07-230d724ee721 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.166139 Sep 17 04:31:35.630160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 156/322] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:35 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:35.678007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:35.686088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-249bdff9-3be9-46f7-ac69-22b2f5ebeadf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.162145 Sep 17 04:31:35.686630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 167/323] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:35 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:36.651966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:36.652570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:36.658640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:36.659109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:36.659548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:36.660238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:36.660687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:36.661127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:36.709127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:36.709894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:36.716728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:36.717333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:36.717839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:36.718637 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:36.719161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:36.719654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:36.799390 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:36.807426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4cfb36e3-8853-42f0-9dd9-c5357ca4fc80 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.159487 Sep 17 04:31:36.807998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 157/324] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:36 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:36.849617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:36.858277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-86ad7057-b09c-498b-b304-0c55c9717fc7 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.154312 Sep 17 04:31:36.858976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 168/325] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:36 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:37.834850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:37.835903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:37.842862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:37.843350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:37.843783 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:37.844507 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:37.844966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:37.845452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:37.881695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:37.882261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:37.890151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:37.890704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:37.891218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:37.891995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:37.892509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:37.893023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:37.978534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:37.986227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3d64c88f-0910-429f-8d82-ad2193218172 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.158024 Sep 17 04:31:37.986794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 158/326] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:37 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:38.025100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:38.033028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-cd43de16-3e38-4910-a3f1-bf36ae1f521e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.155214 Sep 17 04:31:38.033614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 169/327] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:37 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:39.015843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:39.016553 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:39.023925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:39.024600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:39.025189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:39.026061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:39.026623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:39.027150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:39.057697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:39.058353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:39.086911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:39.087417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:39.087871 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:39.088600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:39.089058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:39.089512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:39.107128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:39.178619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:39.186892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-67995d45-a93d-4e3d-a5fb-74bd31aa7203 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.176368 Sep 17 04:31:39.187470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 159/328] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:39 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:39.229521 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:39.237409 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-efef9e9b-c96c-4e0f-9c9d-36c6d3e0035d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.183722 Sep 17 04:31:39.237964 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 170/329] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:39 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:40.211301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:40.212172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:40.219466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:40.220130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:40.220640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:40.221526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:40.222117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:40.222658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:40.261318 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:40.261890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:40.268533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:40.269027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:40.269498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:40.270203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:40.270660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:40.271082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:40.368479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:40.376297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-438fb3ca-bc49-4878-ab4f-ed3531171309 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.170166 Sep 17 04:31:40.377045 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 160/330] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:40 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:40.402127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:40.409764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f0f9f543-8fd4-4fd5-b62c-c24a63c61059 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.152482 Sep 17 04:31:40.410282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 171/331] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:40 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:41.399319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:41.399930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:41.407666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:41.408315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:41.408928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:41.409982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:41.410537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:41.411098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:41.435313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:41.435887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:41.442774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:41.443245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:41.443679 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:41.444386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:41.444839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:41.445292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:41.542224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:41.550308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c8a65491-d0e6-4ad0-ac8b-b68ecc813337 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.154842 Sep 17 04:31:41.550868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 161/332] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:41 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:41.584617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:41.593073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1b605798-cf25-4856-bf79-b870b37cc533 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.162070 Sep 17 04:31:41.593739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 172/333] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:41 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:42.577318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:42.578365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:42.588391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:42.589121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:42.589580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:42.590253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:42.590700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:42.591115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:42.624746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:42.625384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:42.632454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:42.632946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:42.634028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:42.634709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:42.635358 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:42.635787 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:42.721994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:42.730835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0964ec3e-75e9-4db1-94be-2e9e3dcdc33c tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.160075 Sep 17 04:31:42.731399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 162/334] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:42 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:42.766811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:42.777831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5b039943-6d18-4799-8977-bf70206b3e31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.156633 Sep 17 04:31:42.778758 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 173/335] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:42 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:43.754351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:43.755072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:43.761864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:43.762662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:43.763213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:43.763999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:43.764814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:43.765359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:43.806156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:43.807412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:43.814131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:43.814883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:43.815255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:43.815951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:43.816546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:43.819811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:43.908932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:43.917115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7fbd34e5-294c-49cf-838c-0f631573c5f0 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.166961 Sep 17 04:31:43.917683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 163/336] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:43 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:43.954418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:43.962459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-505a7475-765a-4fcb-9f16-3ab47f599798 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.160089 Sep 17 04:31:43.963000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 174/337] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:43 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:44.939889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:44.940483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:44.946284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:44.946753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:44.947224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:44.947888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:44.948361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:44.948833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:44.988115 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:44.988115 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:44.992215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=64 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:44.997661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:44.998241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:44.998908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:44.999799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:45.000432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:45.001034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:45.088480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:45.096398 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-06a3cde3-e6e5-4419-a267-e3b670af3e13 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.160341 Sep 17 04:31:45.096968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 164/338] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:44 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:45.142471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:45.150570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4f6bf12b-53c3-4811-8fb3-9251ce9a610d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.167997 Sep 17 04:31:45.151605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 175/339] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:44 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:46.119168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:46.119973 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:46.126373 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:46.126946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:46.127445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:46.128216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:46.128869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:46.129327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:46.173462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:46.174388 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:46.183620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:46.184316 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:46.184957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:46.185852 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:46.186455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:46.187032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:46.309328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:46.317834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e919961e-fc37-4fd9-b06b-7a6138835123 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.202623 Sep 17 04:31:46.318841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 165/340] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:46 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:46.429323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:46.441140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0be21f93-6935-415f-89db-954d45974b29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.269988 Sep 17 04:31:46.442221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 176/341] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:46 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:47.345058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:47.345685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:47.360810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:47.360810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:47.360810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:47.360810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:47.361201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:47.361201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:47.471874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:47.473231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:47.485701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:47.486198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:47.486625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:47.487512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:47.488905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:47.491909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:47.554908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:47.563443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e021602-6a87-4037-b6c9-3567d3c86e42 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.224687 Sep 17 04:31:47.564335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 166/342] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:47 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:47.569007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5198b50b-0860-472e-b24d-bb4e17962392", "name": "network-vif-plugged", "status": "completed", "tag": "3716a257-cc8a-4082-8240-219d7f1c5cb6"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:47.578870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:47.579345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:47.579768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:47.592608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] Creating event network-vif-plugged:3716a257-cc8a-4082-8240-219d7f1c5cb6 for instance 5198b50b-0860-472e-b24d-bb4e17962392 on hv-cinder-85799 Sep 17 04:31:47.598299 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-21e30394-742a-4143-9c93-c94ad56d05f3 req-461122a0-7a14-4ed8-a6ce-5ab006ef21c1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032801 Sep 17 04:31:47.599535 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 167/343] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:47.700564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:47.708365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9854daf9-9664-4ff9-b45c-1d604547997a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.240360 Sep 17 04:31:47.709565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 177/344] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:47 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:48.591242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:31:48.591798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:31:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 (ip 10.222.0.60) !!! Sep 17 04:31:48.598100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:48.599182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:48.609151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:48.609633 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:48.610064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:48.610734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:48.611202 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:48.611619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:48.732038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:48.732665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:48.739432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:48.739934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:48.740343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:48.741037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:48.741531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:48.741948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:48.743027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:48.751419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-81651ac4-e6b7-4d72-ad4a-b827ec91ffa2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.161586 Sep 17 04:31:48.752020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 168/345] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:48 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:48.881204 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:48.889430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-98e6f41d-e7c2-49a9-99d3-0ef4be8e9672 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.161393 Sep 17 04:31:48.889958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 178/346] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:48 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:49.778636 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:49.779386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:49.786779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:49.787346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:49.787863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:49.788666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:49.789245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:49.789745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:49.912324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:49.912936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:49.919778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:49.920271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:49.920707 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:49.921437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:49.921885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:49.922310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:49.928071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:49.936171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4d3ad429-2664-49a0-8c74-8da95e7d3dc7 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1315 microversion: 2.1 time: 0.164253 Sep 17 04:31:49.936863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 169/347] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:49 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:50.054257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:50.062492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-139be3d5-d67d-40af-b69e-8b9914f332ed tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.154143 Sep 17 04:31:50.063061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 179/348] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:49 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:50.962614 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:50.963311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:50.972339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:50.973064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:50.975063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:50.976024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:50.977164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:50.978227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.084929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:51.085629 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:51.092192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.092681 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.093138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.093889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.094345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.094770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.115090 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:51.123041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b2e5ad11-a6df-472f-a0c4-b747081b59d2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1464 microversion: 2.1 time: 0.167088 Sep 17 04:31:51.123735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 170/349] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:50 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1464 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:51.170940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:51.171417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:51.177690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.178622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.178622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.179333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.179753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.180188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.205683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:31:51.230615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:51.238565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a7a81383-557b-451f-b7d2-60e1e3946a08 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.157552 Sep 17 04:31:51.239185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 180/350] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:51 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:51.260695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:51.299660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-87e03e9a-4f51-40d5-97e1-66a477aed7f9 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "DELETE /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 204 len: 0 microversion: 2.1 time: 0.132364 Sep 17 04:31:51.300413 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 171/351] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:31:51 2022] DELETE /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 17 04:31:51.308761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:51.309406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:51.315648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.316134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.316575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.317342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:51.317793 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:51.318215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:51.450432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:51.458449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f9648470-7e39-4843-b540-baf7363287c2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1470 microversion: 2.1 time: 0.153660 Sep 17 04:31:51.459017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 181/352] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:51 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1470 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:52.265258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:52.265954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:52.272978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:52.273475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:52.273905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:52.274582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:52.275027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:52.275445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:52.421011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:52.428969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-28315002-d5a5-43f0-b32c-7078c34182f0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.168090 Sep 17 04:31:52.429581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 172/353] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:52 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:52.484321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:52.484911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:52.491779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:52.492270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:52.492705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:52.493430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:52.493881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:52.494308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:52.628653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:52.636615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-22bee164-f3ea-4df7-bca3-9b69c0f38e6a tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1470 microversion: 2.1 time: 0.158262 Sep 17 04:31:52.637166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 182/354] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:52 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1470 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:52.734486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1718422253", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-623666681", "security_groups": [{"name": "tempest-secgroup-smoke-2116056742"}], "block_device_mapping_v2": [{"uuid": "60182b5d-bc1a-403c-8972-625aa7fb815e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e322731d-9c63-4618-8051-e8dea6bb2db6"}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:52.858836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] validate_networks() for [('e322731d-9c63-4618-8051-e8dea6bb2db6', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:31:53.228483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:31:53.228906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:31:53.229482 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:31:53.229862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:31:53.230443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:31:53.240154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.240692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.241474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.242206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.242658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.243084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.262242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:31:53.268382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for user ff241cbd3f7c47328ac5ef2ed4c66e18 and project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:31:53.335684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:31:53.338413 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='60182b5d-bc1a-403c-8972-625aa7fb815e',volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:31:53.453713 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:53.454332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:53.470949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.471432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.471857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.472557 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.472997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.473811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.516907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-64d3ca3d-f587-46af-b735-bf124d2e52d6 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.786325 Sep 17 04:31:53.517581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 173/355] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:31:52 2022] POST /compute/v2.1/servers => generated 406 bytes in 788 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:31:53.528915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-eca39cbd-381e-4658-904b-c8b1c1bd8bd7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:53.529917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-eca39cbd-381e-4658-904b-c8b1c1bd8bd7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:53.599742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-eca39cbd-381e-4658-904b-c8b1c1bd8bd7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:53.607063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-eca39cbd-381e-4658-904b-c8b1c1bd8bd7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1094 microversion: 2.1 time: 0.081993 Sep 17 04:31:53.608299 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 174/356] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:53 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1094 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:53.660399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:53.661237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:53.666081 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:53.667532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.668042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.668474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.669200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:53.669983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:53.670563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:53.676220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e19455e4-9497-4f49-affa-11e7c2d5f9a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.226260 Sep 17 04:31:53.677176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 183/357] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:53 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:53.825144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:53.834765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7853a9f1-48d9-4f56-9e1a-6fd3e3b9d2f2 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1470 microversion: 2.1 time: 0.178497 Sep 17 04:31:53.849142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 175/358] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:53 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1470 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:54.170461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5198b50b-0860-472e-b24d-bb4e17962392", "name": "network-vif-deleted", "tag": "3716a257-cc8a-4082-8240-219d7f1c5cb6"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:54.185791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.186276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.186737 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.198736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] Creating event network-vif-deleted:3716a257-cc8a-4082-8240-219d7f1c5cb6 for instance 5198b50b-0860-472e-b24d-bb4e17962392 on hv-cinder-85799 Sep 17 04:31:54.203678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-e9859eb6-0d1e-49c1-b7ea-88c5c42c0f5b req-1ba8b0fd-e641-4c7e-8547-ec7968469667 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038092 Sep 17 04:31:54.204607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 184/359] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:54.631204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:54.631851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:54.638594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.639171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.639645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.640530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.641042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.641573 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.700944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:54.701801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:54.705933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:54.708403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.708972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.709505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.710285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.710812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.711332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.713680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-32f34b77-ce1b-4c47-9367-2ea760c3cfbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1192 microversion: 2.1 time: 0.086585 Sep 17 04:31:54.714250 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 176/360] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:54 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1192 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:54.843943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:54.853554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-cbef6d48-a2f2-403b-9a35-cb62187bffb2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.157339 Sep 17 04:31:54.854213 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 185/361] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:54 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:54.869386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:54.869386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:54.869386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=80,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:54.869386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.869386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.870057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.870057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.870057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.870304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.910316 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:54.910937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:54.911383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:54.911888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lazy-loading 'fault' on Instance uuid 5198b50b-0860-472e-b24d-bb4e17962392 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:31:54.963309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5817b0d9e6904be2b0ea482746d3d70f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6619450eef5a4a5b8d3b6a45fe412788', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:54.971201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-032c0fa7-1d6d-43e4-944f-6f39c3e48524 tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 200 len: 1305 microversion: 2.1 time: 0.115106 Sep 17 04:31:54.985115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 177/362] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:54 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 1305 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:55.737309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:55.737776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:55.747468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:55.748201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:55.748972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:55.750237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:55.750918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:55.751772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:55.843687 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:55.852263 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1ec4dbef-3410-4332-ad40-c23cf19b69f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1210 microversion: 2.1 time: 0.119251 Sep 17 04:31:55.852834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 186/363] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:55 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1210 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:55.877519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:55.878407 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:55.884575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:55.885052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:55.885679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:55.886365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:55.886814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:55.887230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:56.002065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:56.006047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] [instance: 5198b50b-0860-472e-b24d-bb4e17962392] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:56.015054 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:56.015742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:56.016203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:56.016966 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:56.018021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:56.018189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:56.035147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] HTTP exception thrown: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Request method failure captured: Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 HTTP/1.1 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU2NcIgJvg_ZVz-bmDZ04kXWHprcobeyZWspSbvWy7xlNK7BkSs3D7HT0cmGE4sKyIAx_mw_DrVFfYKsa9clcDJX7R70zKx1oVLFMun1ONRCqZcrvfdewYPhbk7_24O6L8meNLgEaVrpZnRA_0neG4SJ9h7vTmAv-Jqs_vsU7RKkpB0 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 6619450eef5a4a5b8d3b6a45fe412788 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesNegativeTest-1658295876 Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:31:56.036845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/6619450eef5a4a5b8d3b6a45fe412788"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/6619450eef5a4a5b8d3b6a45fe412788"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_6619450eef5a4a5b8d3b6a45fe412788"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/6619450eef5a4a5b8d3b6a45fe412788"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesNegativeTest-1658295876 Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 6619450eef5a4a5b8d3b6a45fe412788 Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesNegativeTest-1658295876 Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesNegativeTest-1658295876-project Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 5817b0d9e6904be2b0ea482746d3d70f Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesNegativeTest-1658295876-project Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. Sep 17 04:31:56.038820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '5198b50b-0860-472e-b24d-bb4e17962392'} Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:31:56.039756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. Sep 17 04:31:56.042371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:31:56.045055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] Returning 404 to user: Instance 5198b50b-0860-472e-b24d-bb4e17962392 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:31:56.045055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6dc7e358-ed32-4c31-90c8-9717e36957fc tempest-VolumesNegativeTest-1658295876 tempest-VolumesNegativeTest-1658295876-project] 10.222.0.60 "GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392" status: 404 len: 111 microversion: 2.1 time: 0.045863 Sep 17 04:31:56.045055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 187/364] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:55 2022] GET /compute/v2.1/servers/5198b50b-0860-472e-b24d-bb4e17962392 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:31:56.049660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:56.061345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f78d982c-a212-4dd2-b4ce-d1aba69da240 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.187564 Sep 17 04:31:56.062765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 178/365] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:55 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:56.878329 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:56.888025 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:56.910590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:56.911089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:56.911641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:56.912372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:56.912981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:56.917620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:56.921596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f28c067-5b02-4cb7-8284-c59d2f694528", "tag": "fa6be46f-8a19-4a8d-84f9-67c67c4d32be"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:31:56.952000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:56.955813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:31:56.956686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:56.957184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:57.002311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] Creating event network-changed:fa6be46f-8a19-4a8d-84f9-67c67c4d32be for instance 8f28c067-5b02-4cb7-8284-c59d2f694528 on hv-cinder-85799 Sep 17 04:31:57.011054 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-ecdeb8ae-2a6e-4660-9ac7-8d3860c459f3 req-cf1ee5ab-c0fb-47ee-8b45-a6c48ed04696 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100397 Sep 17 04:31:57.011666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 179/366] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:31:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:31:57.090730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:57.091722 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:57.099412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:57.099975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:57.100471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:57.101503 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:57.102789 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:57.103278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:57.125233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:57.134754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2e63868b-c5ce-4728-bce6-379dab582824 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.261524 Sep 17 04:31:57.135475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 188/367] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:56 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:57.263003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:57.272271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-50aea163-4e82-4789-83aa-4365082a2719 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.185715 Sep 17 04:31:57.272908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 180/368] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:57 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:31:58.162087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:58.167761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:58.171778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:31:58.171946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:31:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 (ip 10.222.0.60) !!! Sep 17 04:31:58.180044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:58.180717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:58.181377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:58.182260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:58.182717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:58.183284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:58.300855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:58.301661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:58.308743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:58.309475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:58.310342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:58.311246 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:58.312060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:58.312773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:58.489282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:58.498681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-73de4f0a-d89b-447b-a9a8-091c938d5f98 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.207066 Sep 17 04:31:58.499600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 181/369] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:58 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:58.541004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:58.553187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-de8819fa-ec66-4dc9-9959-783c7f14f797 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.393382 Sep 17 04:31:58.554277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 189/370] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:58 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:59.521320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:59.521922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:59.529133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:59.529820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:59.530426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:59.531400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:59.531990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:59.532558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:59.578579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:31:59.580041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:31:59.588624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:59.589680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:59.590324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:59.591207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:31:59.592340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:31:59.592952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:31:59.683267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:59.692744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cd245c96-0a04-4557-847c-165d15c6be38 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1522 microversion: 2.1 time: 0.175252 Sep 17 04:31:59.693366 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 182/371] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:59 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:31:59.809900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:31:59.817535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8d36ec6f-a7af-4682-b08a-e7b596119da2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.244256 Sep 17 04:31:59.818134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 190/372] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:31:59 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:00.716137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:00.716756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:00.723460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.723953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.724386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:00.725105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.725569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.725968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:00.840417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:00.841241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:00.848700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.849218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.849851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:00.850750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.851399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.851825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:00.867556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:00.875144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0e9851eb-02f5-4546-ad70-fcc526291aeb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1502 microversion: 2.1 time: 0.162827 Sep 17 04:32:00.875550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 183/373] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:00 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1502 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:00.902435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:00.903041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:00.909193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.909864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.910309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:00.910980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:00.911436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:00.911855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:01.003744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:01.012107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-92ffa9f6-7757-43b2-be65-69fdc1ae95ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.175841 Sep 17 04:32:01.012689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 191/374] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:00 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:01.050351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:01.058113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-82ffd24a-c898-435a-bc20-86f22788590e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1515 microversion: 2.1 time: 0.164825 Sep 17 04:32:01.058686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 184/375] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:00 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1515 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:02.035402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:02.036031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:02.043024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.043516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.044008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.044708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.045212 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.045640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.179800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:02.189748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b2414ffe-ce30-4ff1-a3dc-9b7f603326f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.158256 Sep 17 04:32:02.190664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 192/376] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:02 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:02.513270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:02.515556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:02.522079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.522580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.523009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.523702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.524315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.524897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.548117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'flavor' on Instance uuid c0837134-994e-4402-a813-3bacee6f4853 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:32:02.664077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-511ad72c-03a0-4c59-84d9-9fcc73f3d87b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/action" status: 200 len: 32 microversion: 2.1 time: 0.154553 Sep 17 04:32:02.664999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 185/377] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:32:02 2022] POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/action => generated 32 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:32:02.681028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:02.681882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:02.688958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.689708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.690310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.691291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:02.692046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:02.692690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:02.834702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:02.846668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-13661b3a-0feb-40a0-8b46-6a37c86d06c6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.171498 Sep 17 04:32:02.847500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 193/378] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:02 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:03.213775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:03.214458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:03.221177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:03.221663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:03.222103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:03.222794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:03.223620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:03.224043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:03.368833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:03.501862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:03.510184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-69f8e4ac-0a91-4708-ba10-1b8de983605d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.300778 Sep 17 04:32:03.510737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 186/379] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:03 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:03.874252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:03.875252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:03.886280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:03.887113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:03.887832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:03.889003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:03.889811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:03.890543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:04.026103 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:04.034602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d68a763d-800b-4a25-b4da-efaab4959ccc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.166852 Sep 17 04:32:04.035170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 194/380] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:03 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:04.537140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:04.538177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:04.541514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=8 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:04.549295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:04.549774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:04.550205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:04.550892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:04.551345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:04.551760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:04.685151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:04.693598 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a2976317-9adc-41e9-adf0-c1c5373b3bfe tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.163069 Sep 17 04:32:04.694125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 187/381] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:04 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:05.061226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:05.062051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:05.072614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:05.073488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:05.074208 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:05.075345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:05.076098 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:05.076804 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:05.208187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:05.216597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f807a31a-ede9-4180-9bc8-87c58dc7ed8c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.162052 Sep 17 04:32:05.217149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 195/382] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:05 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:05.716242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:05.716878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:05.723847 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:05.724334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:05.724781 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:05.725520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:05.725934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:05.726395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:05.729266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:05.860578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:05.868450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c82712d0-49c8-4c2f-9f9b-044a3e1e8970 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.156107 Sep 17 04:32:05.868962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 188/383] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:05 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:06.244532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:06.245509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:06.254225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:06.254714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:06.255153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:06.255847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:06.256304 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:06.256730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:06.390292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:06.398370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e94e62c4-fee3-4984-8113-ebeabb51bcdd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.160656 Sep 17 04:32:06.398990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 196/384] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:06 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:06.892241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:06.892820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:06.898912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:06.899403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:06.899838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:06.900553 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:06.901013 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:06.901829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:07.037875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:07.045546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-25eae64a-50de-4efa-b49a-23072f02a20f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.157902 Sep 17 04:32:07.046105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 189/385] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:06 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:07.422100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:07.422739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:07.429722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:07.430208 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:07.430643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:07.431338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:07.431794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:07.432217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:07.575441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:07.583910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aea059dc-6ee6-4f2b-8eb4-d8b8756b5b6d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.165940 Sep 17 04:32:07.584509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 197/386] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:07 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:08.067821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:08.068454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:08.074800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:08.075291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:08.075758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:08.076498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:08.077399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:08.077399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:08.103943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-162617489", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "networks": [{"uuid": "bb95167b-27c5-4a3a-8d9b-1b9930df9108"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:08.177381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] validate_networks() for [('bb95167b-27c5-4a3a-8d9b-1b9930df9108', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:32:08.226188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:08.234363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-67197ea5-4b91-4931-8266-e9938a997570 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.170466 Sep 17 04:32:08.234907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 190/387] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:08 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:08.608273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:08.608995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:08.615738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:08.616280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:08.616754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:08.617518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:08.617977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:08.618442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:08.761752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:08.770134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a554cb18-3ed2-4a07-af41-62d138757072 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.165763 Sep 17 04:32:08.770684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 191/388] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:08 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:08.991254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:32:08.991732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:32:08.992156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:32:08.992551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:32:08.992981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:32:09.001794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.002267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.002690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.003391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.003833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.004244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.021125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Getting quotas for project 9d47a53afdbf404a829c119992fcb643. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:32:09.025170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Getting quotas for user 166d99888375478e9cd8bf8dcb9d4190 and project 9d47a53afdbf404a829c119992fcb643. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:32:09.036125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:32:09.038533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:32:09.086063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ed97fbe6-11b7-478a-a29e-c6b59bfacf18 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.986076 Sep 17 04:32:09.086705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 198/389] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:32:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 987 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 17 04:32:09.098498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3c7c6e87-5c63-4935-b90a-823c582c91b5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:09.099244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3c7c6e87-5c63-4935-b90a-823c582c91b5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:09.151396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3c7c6e87-5c63-4935-b90a-823c582c91b5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:09.159081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3c7c6e87-5c63-4935-b90a-823c582c91b5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1218 microversion: 2.1 time: 0.064977 Sep 17 04:32:09.159746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 192/390] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:09 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1218 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:09.257559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:09.258176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:09.265258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.265732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.266130 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.266817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.267261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.267653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.433824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:09.445201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9f517744-32d1-49ce-8783-75945e886919 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.191582 Sep 17 04:32:09.445914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 199/391] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:09 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:09.796171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:09.796843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:09.803983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.804473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.804891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.805619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:09.806091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:09.807051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:09.940309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:09.948483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-503e1d0e-8e2c-4225-ac71-c793080e387d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.158966 Sep 17 04:32:09.949042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 193/392] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:09 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:10.181892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:10.182909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:10.190260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.190836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.191345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:10.192252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.192776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.193293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:10.258715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:10.269625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f59ecbd1-7233-4327-b16e-ad7f6cee07a2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1270 microversion: 2.1 time: 0.091761 Sep 17 04:32:10.270617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 200/393] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:10 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1270 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:10.469289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:10.469700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:10.476695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.477588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.478789 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:10.479682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.480265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.480700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:10.686763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:10.696989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ea518635-cdbf-492f-8e8a-95d4e14b0a0f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.232369 Sep 17 04:32:10.697706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 194/394] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:10 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:10.972147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:10.973563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:10.983363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.983952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.984484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:10.985294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:10.985901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:10.986449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.167522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:11.175470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-012206b4-c663-40a7-8750-f2def71f2be3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.207453 Sep 17 04:32:11.176312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 201/395] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:10 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:11.292411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:11.293060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:11.299648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:11.300130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:11.300559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.301288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:11.301737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:11.302154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.445908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:11.454677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d27edfb3-3b15-402b-91b4-ab84fdd72fe5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.166184 Sep 17 04:32:11.465726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 195/396] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:11 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:11.526837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f28c067-5b02-4cb7-8284-c59d2f694528", "name": "network-vif-plugged", "status": "completed", "tag": "fa6be46f-8a19-4a8d-84f9-67c67c4d32be"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:11.537277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:11.537861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:11.538309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.551576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] Creating event network-vif-plugged:fa6be46f-8a19-4a8d-84f9-67c67c4d32be for instance 8f28c067-5b02-4cb7-8284-c59d2f694528 on hv-cinder-85799 Sep 17 04:32:11.556968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-0ebb42d8-62d4-4ae4-8de4-126c4238be56 req-1011b533-f551-46dd-aeff-bf6fa8647458 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034211 Sep 17 04:32:11.557816 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 202/397] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:11.722006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:11.722776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:11.730095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:11.730689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:11.731167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.732347 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:11.732866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:11.733414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:11.877609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:11.886453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fa19ed29-760e-4763-af2a-f28486c3fa6f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.169072 Sep 17 04:32:11.887202 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 196/398] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:11 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:12.056550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "564a5f73-27b3-43f1-9c5f-3c69ecc186a5", "tag": "9b74c853-353e-4c20-a37a-ebb673727be5"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:12.066558 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.067131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.067549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.078612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] Creating event network-changed:9b74c853-353e-4c20-a37a-ebb673727be5 for instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 on hv-cinder-85799 Sep 17 04:32:12.084116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-6674dbab-76ff-4753-a647-c81342d0068c req-dcf7fec3-b02a-4b49-81e1-d0f5ecbfc55b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031959 Sep 17 04:32:12.084822 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 203/399] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:12.202775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:12.203780 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:12.214090 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.214897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.215626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.216778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.217582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.218298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.350603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:12.362703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b03c32f7-a9f4-4b06-a4e8-5b06c5214870 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.166208 Sep 17 04:32:12.363328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 197/400] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:12 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:12.478049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:12.478749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:12.485463 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.486107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.486584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.487296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.487745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.488168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.636673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:12.644975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2d29252c-e0b1-4cc5-af94-f52dfeb5bd5c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.171181 Sep 17 04:32:12.645747 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 204/401] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:12 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:12.914790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:12.915655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:12.926125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.926906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.927632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:12.928800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:12.929566 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:12.930282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:13.075951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:13.085466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-26772bba-fceb-45c8-9eab-0b1d5417023f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1266 microversion: 2.1 time: 0.177395 Sep 17 04:32:13.086041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 198/402] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:12 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:13.386174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:13.386919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:13.394769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:13.395260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:13.395693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:13.396488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:13.396975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:13.397442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:13.604717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:13.612863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c6d83ce0-f587-43c5-9243-e532ca018c01 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.230619 Sep 17 04:32:13.613697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 205/403] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:13 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:13.667944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:13.668542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:13.675066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:13.675629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:13.676076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:13.676768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:13.677279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:13.677704 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:13.809531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:13.817510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0b62d2d7-07b7-4766-b51e-a29ed92748fe tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.153514 Sep 17 04:32:13.818080 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 199/404] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:13 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:14.112754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:14.113487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:14.120755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.121365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.121874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.122674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.123196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.123696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.264956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:14.274841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b6c8482e-f8f7-4243-90bd-b4dfc030f7d1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1454 microversion: 2.1 time: 0.168328 Sep 17 04:32:14.275428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 206/405] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:14 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1454 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:14.297882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:14.298445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:14.305225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.305773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.306216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.306914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.307381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.307802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.454906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:14.462891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-23127610-65f7-40cf-bd32-8972252c231a tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1454 microversion: 2.1 time: 0.169109 Sep 17 04:32:14.463452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 200/406] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:14 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1454 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:14.636423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:14.637260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:14.666711 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.667867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.669012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.670475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.671101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.671720 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.678082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:14.841678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:14.842000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:14.849018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.849569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.850141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.851009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:14.851497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:14.851906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:14.889794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:14.898636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-39681593-571f-419c-8afd-5e09f0f77342 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.266242 Sep 17 04:32:14.899403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 207/407] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:14 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:15.194131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:15.202694 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-443bc53c-b27a-450e-8da6-f5008bebde8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.366601 Sep 17 04:32:15.203333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 201/408] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:14 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:15.922634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:15.923235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:15.925595 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:15.930965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:15.931459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:15.931894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:15.932587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:15.933041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:15.933500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:16.067342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:16.075413 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-147d7526-d3f2-4a39-8d69-d63aed3dd623 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.156768 Sep 17 04:32:16.076061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 208/409] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:15 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:16.227049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:16.227645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:16.233953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:16.234436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:16.234868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:16.235557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:16.236013 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:16.236544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:16.433677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:16.441495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-523895bd-9e5d-4e35-912f-a68a22539a6b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.218750 Sep 17 04:32:16.442295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 202/410] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:16 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:16.592806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f28c067-5b02-4cb7-8284-c59d2f694528", "tag": "fa6be46f-8a19-4a8d-84f9-67c67c4d32be"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:16.603543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:16.604013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:16.604462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:16.619268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] Creating event network-changed:fa6be46f-8a19-4a8d-84f9-67c67c4d32be for instance 8f28c067-5b02-4cb7-8284-c59d2f694528 on hv-cinder-85799 Sep 17 04:32:16.624866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-461dfbfe-e970-4de4-b56c-06061725660f req-ec566870-28c4-448a-88f9-fc857943d32c service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035969 Sep 17 04:32:16.625495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 209/411] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:17.099821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:17.100437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:17.106839 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:17.107336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:17.107768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:17.108464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:17.109022 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:17.109718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:17.245822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:17.254334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f8bef664-56dd-4b41-8337-4bcba3b4d5d9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.159368 Sep 17 04:32:17.255195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 203/412] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:17 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:17.464952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:17.465690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:17.473263 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:17.473752 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:17.474189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:17.474876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:17.475331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:17.475758 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:17.618571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:17.627467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3c4a9671-6fc3-4d26-afa5-aae099eb78c5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.166563 Sep 17 04:32:17.627970 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 210/413] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:17 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:18.277757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:18.278327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:18.284711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:18.285229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:18.285670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:18.286386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:18.286857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:18.287281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:18.423067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:18.432180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c60f90ea-2d7b-4779-a44c-c4a4fc83484d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.158357 Sep 17 04:32:18.432861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 204/414] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:18 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:18.650284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:18.650916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:18.657909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:18.658408 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:18.658848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:18.659578 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:18.660031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:18.660455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:18.794181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:18.802402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-40b79c26-10ac-4e25-9119-e2c22f210677 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.156326 Sep 17 04:32:18.802966 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 211/415] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:18 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:19.231960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:19.232764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:19.235701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:19.242383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.242858 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.243281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.243949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.244389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.244800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.268244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c4735452-cf02-4b6e-a277-c3043cb2f819 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/ips" status: 200 len: 148 microversion: 2.1 time: 0.042806 Sep 17 04:32:19.268788 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 205/416] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:32:19 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/ips => generated 148 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:19.457676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:19.458594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:19.465104 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:19.466216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:19.468308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.469140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.469904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.470840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.471594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.472397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.676214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:19.684746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-caa320bf-139b-4424-ac3f-e5baf4a96acb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.232036 Sep 17 04:32:19.685374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 212/417] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:19 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:19.830152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:19.831191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:19.841317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.842108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.842812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.843936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:19.844678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:19.845638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:19.849048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:19.981214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:19.992826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f1615a80-d702-4d13-907e-8633b95b3175 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.169397 Sep 17 04:32:19.993681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 206/418] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:19 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:20.663455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "02feb35d-2600-4285-bc1e-2644b9a0d936"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:20.665941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:20.672575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:20.673065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:20.673537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:20.674231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:20.674693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:20.675117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:20.707922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:20.708510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:20.715211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:20.715692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:20.716125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:20.716813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:20.717300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:20.717718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:20.844752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:20.853701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7ada6097-2c6b-49de-aa16-d421f46f4119 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.149580 Sep 17 04:32:20.854199 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 207/419] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:20 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:21.024724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:21.025458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:21.032741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:21.033352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:21.033856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:21.034652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:21.035171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:21.035661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:21.170737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:21.178903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3160c499-54f5-474d-a381-9630116be278 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.159122 Sep 17 04:32:21.179445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 208/420] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:21 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:21.776687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "564a5f73-27b3-43f1-9c5f-3c69ecc186a5", "name": "network-vif-plugged", "status": "completed", "tag": "9b74c853-353e-4c20-a37a-ebb673727be5"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:21.790667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:21.791415 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:21.792103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:21.805434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] Creating event network-vif-plugged:9b74c853-353e-4c20-a37a-ebb673727be5 for instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 on hv-cinder-85799 Sep 17 04:32:21.807042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-41bf3244-0118-4feb-a997-d2f7d5ca15a7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.147163 Sep 17 04:32:21.808032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 213/421] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:32:20 2022] POST /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments => generated 194 bytes in 1149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:21.812383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-76847a7c-7921-4355-9379-4e1402ff1b1a req-9143ea28-01cd-4dcb-9787-71e5e9853120 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040381 Sep 17 04:32:21.813644 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 209/422] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:32:21.876862 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:21.877775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:21.884079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:21.884561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:21.884998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:21.885705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:21.886157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:21.886576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:22.019608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:22.027563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1aa734b3-078a-4396-a06a-6c26770bff1a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.155591 Sep 17 04:32:22.028122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 214/423] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:21 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:22.207155 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:22.208154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:22.215511 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:22.216000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:22.216440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:22.217113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:22.217632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:22.218057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:22.350319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:22.360653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d3324005-d3a3-42a8-9eff-82d256dfd2b2 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.160427 Sep 17 04:32:22.361515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 210/424] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:22 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:23.051032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:23.051441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:23.058187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:23.058688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:23.059202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:23.060124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:23.060575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:23.061004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:23.207124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:23.214992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-811f5076-690d-4917-9203-61ccfccbddde tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.168208 Sep 17 04:32:23.215552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 215/425] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:23 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:23.384533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:23.385316 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:23.391734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:23.392194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:23.392973 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:23.393628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:23.394103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:23.394529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:23.530351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:23.539104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a66bb737-f415-41a0-a9cf-ee87ab1a48d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.158904 Sep 17 04:32:23.540019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 211/426] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:23 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:24.243454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:24.244326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:24.255255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.256052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.256723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.257456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.257912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.258332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.397127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:24.405188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b710d2f5-7487-42ef-a101-fd54c0e5ec3c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1501 microversion: 2.1 time: 0.169377 Sep 17 04:32:24.406023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 216/427] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:24 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1501 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:24.428257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8da9cd6f-d034-460a-804b-7a04ec5546ae"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:24.431357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:24.438508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.439035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.439521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.440746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.441339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.441918 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.466509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] HTTP exception thrown: Cannot 'attach_volume' instance c0837134-994e-4402-a813-3bacee6f4853 while it is in vm_state rescued Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Request method failure captured: Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/os-volume_attachments HTTP/1.1 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 74 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU1VyO3YwjgPsO3zAE4FZmsyWEcgb5B0Oy5RcSv7jAJnr5DJfg3c80tKcUCh5_S8hLjEcJ5qglAE8bqRQMN3PZ86lwmiGwGAxYPEQcqx_A0AYCoaReV-dg5nm3Mg-oVytdBrAl3CriJDBmv83kbBOhg2wmCsC9XwgpE4vLYHXFOTWwg Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:32:24.468321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 5bb515269d1a45f594f3d297c7fd267e Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {"volumeAttachment": {"volumeId": "8da9cd6f-d034-460a-804b-7a04ec5546ae"}} Sep 17 04:32:24.469863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Cannot 'attach_volume' instance c0837134-994e-4402-a813-3bacee6f4853 while it is in vm_state rescued Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'server_id': 'c0837134-994e-4402-a813-3bacee6f4853', 'body': {'volumeAttachment': {'volumeId': '8da9cd6f-d034-460a-804b-7a04ec5546ae'}}} Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c0837134-994e-4402-a813-3bacee6f4853 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:32:24.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance c0837134-994e-4402-a813-3bacee6f4853 while it is in vm_state rescued Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:32:24.471852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance c0837134-994e-4402-a813-3bacee6f4853 while it is in vm_state rescued Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Returning 409 to user: Cannot 'attach_volume' instance c0837134-994e-4402-a813-3bacee6f4853 while it is in vm_state rescued {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-220da3d9-5b4b-4a96-b197-75ff82b4e4b6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.046067 Sep 17 04:32:24.473384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 212/428] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:32:24 2022] POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/os-volume_attachments => generated 152 bytes in 47 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 17 04:32:24.479422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:24.479940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:24.485940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.486408 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.486829 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.487495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.487935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.488347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.556268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-10b6bcbb-5415-4f61-ab80-2ff03d805c80 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/action" status: 202 len: 0 microversion: 2.1 time: 0.080636 Sep 17 04:32:24.556890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 217/429] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:32:24 2022] POST /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:32:24.563896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:24.564502 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:24.565359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:24.566057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:24.570901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.571377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.571801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.572477 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.572925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.573059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.573375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.573548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.573984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.574659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:24.575116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:24.575547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:24.707676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:24.707851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:24.715740 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a320fc85-f0e1-4679-aab1-47bbaac45606 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1318 microversion: 2.1 time: 0.155798 Sep 17 04:32:24.716276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fa0ce9c7-a6fb-4d40-97a6-1737f03c0867 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.154906 Sep 17 04:32:24.716409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 213/430] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:24 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:24.717023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 218/431] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:24 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:25.744264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:25.745341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:25.746010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:25.746207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:25.750388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=78 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:25.756021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:25.756526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:25.756743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:25.757014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:25.757219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:25.757479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:25.757934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:25.758152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:25.758366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:25.758603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:25.758784 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:25.759021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:25.891911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:25.900484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1e1fd0b9-3503-4895-b9c4-84b4bb9a2a5e tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1507 microversion: 2.1 time: 0.162943 Sep 17 04:32:25.901052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 214/432] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:25 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1507 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:25.928986 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:25.937041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a365ee75-f893-4283-95c9-8acb0e505920 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.199162 Sep 17 04:32:25.953141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 219/433] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:25 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:26.962674 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:26.963247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:26.970054 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:26.970537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:26.970951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:26.971661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:26.972126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:26.972913 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:27.115749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:27.123798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7ee8c1b1-ccb1-47df-a8cb-57e84def9f27 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.165286 Sep 17 04:32:27.124348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 215/434] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:26 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:27.509695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:27.510270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:27.516826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:27.517330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:27.517728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:27.518657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:27.519128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:27.519620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:27.545063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-cd1a0f7e-392a-49a2-8c92-a68d1ae707f2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/ips" status: 200 len: 149 microversion: 2.20 time: 0.039183 Sep 17 04:32:27.546411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 220/435] 10.222.0.60 () {60 vars in 1289 bytes} [Sat Sep 17 04:32:27 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:32:28.152124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:28.153033 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:28.163027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:28.163866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:28.164638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:28.165815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:28.166561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:28.167256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:28.304761 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:28.313695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d9b103c4-f614-43ac-9520-e0703107f415 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.168417 Sep 17 04:32:28.318098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 216/436] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:28 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:28.330127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bdab3c7f-53b9-471d-a95d-ffd653a487e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: .version_select at 0x7fbc4f4ebca0>, body: {"keypair": {"name": "tempest-keypair-51696387", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGypConcFIOtPqDnDQWqNuVQNHwn7brYfGDyjdS8uZ84UkapFKPYIwFUJQ7ojWoVrNX42T8Ebn3EUzmyj65rDIozK+gl1SBJgx4/lEikAJpvIwNXZvm0wAEpO0zvF8+cXA=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:28.345998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-bdab3c7f-53b9-471d-a95d-ffd653a487e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:32:28.350243 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-bdab3c7f-53b9-471d-a95d-ffd653a487e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:32:28.375326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bdab3c7f-53b9-471d-a95d-ffd653a487e2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.049076 Sep 17 04:32:28.375921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 221/437] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:32:28 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:29.340695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:29.341318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:29.347320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:29.348161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:29.348591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:29.349308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:29.349756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:29.350178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:29.489461 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:29.498209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0f4400a6-0993-4c84-9d4d-9bd6eb00709d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.161399 Sep 17 04:32:29.499185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 217/438] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:29 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:29.735939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1244604358", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5d741fbf-1171-42a3-8de2-aef8bbcc1581"}], "security_groups": [{"name": "tempest-securitygroup--122659239"}], "key_name": "tempest-keypair-51696387", "user_data": "}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:29.840179 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] validate_networks() for [('5d741fbf-1171-42a3-8de2-aef8bbcc1581', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:32:30.492147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:32:30.492576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:32:30.493054 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:32:30.493503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:32:30.494037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:32:30.504251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.504739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.505211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:30.505937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.506392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.506816 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:30.522272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:30.523614 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:30.527206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:32:30.529669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.530153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.530714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:30.531409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.531854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.532251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:32:30.532369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:30.546031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:32:30.548912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:32:30.599648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b291f8ad-0e14-4d04-9027-7e074bf184e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.868634 Sep 17 04:32:30.601096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 222/439] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:32:29 2022] POST /compute/v2.1/servers => generated 401 bytes in 871 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:32:30.612344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8fcb56d2-3db6-403f-9247-0ce968bef7aa tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:30.613112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8fcb56d2-3db6-403f-9247-0ce968bef7aa tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:30.680232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:30.680503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8fcb56d2-3db6-403f-9247-0ce968bef7aa tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:30.688192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8fcb56d2-3db6-403f-9247-0ce968bef7aa tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1240 microversion: 2.1 time: 0.079883 Sep 17 04:32:30.688513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-66d9d60f-1f1b-4913-a921-6cc4d82c1f31 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1509 microversion: 2.1 time: 0.170310 Sep 17 04:32:30.688758 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 223/440] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:30 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1240 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:30.689444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 218/441] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:30 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1509 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:30.965160 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:30.965923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:30.973960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.974458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.974894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:30.975589 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:30.976130 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:30.976546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.000660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lazy-loading 'flavor' on Instance uuid 0043b5dd-c019-4a76-b2d0-4c68982cac21 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:32:31.020772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:31.189779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a4eab28b-2ce3-4a76-befc-2b69470962d0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/action" status: 202 len: 0 microversion: 2.20 time: 0.228388 Sep 17 04:32:31.190351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 224/442] 10.222.0.60 () {62 vars in 1319 bytes} [Sat Sep 17 04:32:30 2022] POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/action => generated 0 bytes in 230 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:32:31.198794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:31.199404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:31.205547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.206028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.206442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.207115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.207565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.207976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.341638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:31.350201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:31.351583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:31.352705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6c982a18-bc24-440a-b38e-5af48c8123d7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.157838 Sep 17 04:32:31.353280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 219/443] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:31 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:31.711711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:31.712376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:31.712478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:31.712760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:31.719120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.719272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.719603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.719774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.720011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.720130 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.720738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.720881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:31.721224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.721393 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:31.721651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.721796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:31.790622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:31.799085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-da837a0b-e5d6-4ff2-a1ae-f86d8fc863e9 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1244 microversion: 2.1 time: 0.090948 Sep 17 04:32:31.799664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 225/444] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:31 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1244 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:31.868245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:31.877018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-01237f81-3e2b-4ee1-a96b-1f3c0a6c2086 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1515 microversion: 2.1 time: 0.169313 Sep 17 04:32:31.877699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 220/445] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:31 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1515 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:32.383698 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:32.384874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:32.391594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:32.392101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:32.392503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:32.393400 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:32.394036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:32.394449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:32.533939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:32.542600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:32.544023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:32.545230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-de2ea37a-d328-420b-9806-6e54bbb824a9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.165561 Sep 17 04:32:32.545780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 226/446] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:32 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:32.828773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:32.829994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:32.838655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:32.839136 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:32.839566 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:32.840290 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:32.840735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:32.841148 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:32.904855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:32.912925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6ee48704-80af-4719-9b7e-76ba88d0eecd tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1292 microversion: 2.1 time: 0.091587 Sep 17 04:32:32.913554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 221/447] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:32 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1292 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:33.571002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:33.571803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:33.579289 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:33.579780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:33.580474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:33.581322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:33.581917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:33.582347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:33.788715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:33.797799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:33.799227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:33.800393 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d1583cdf-c580-4283-9f21-e34c4e9588b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.233851 Sep 17 04:32:33.800924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 227/448] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:33 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:33.936025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:33.936625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:33.944054 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:33.944590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:33.945027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:33.945752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:33.946207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:33.946628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:34.015319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:34.023134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b094bc68-4ec5-4141-a853-8b3c1033bea8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1298 microversion: 2.1 time: 0.091052 Sep 17 04:32:34.023687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 222/449] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:33 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1298 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:34.302218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d8568eaa-d711-4115-af49-7de8b4cfeaa5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: .version_select at 0x7fbc4f559f70>, body: {"keypair": {"name": "tempest-keypair-1018766987", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKqyAyjfXXKuPugeF2yBXKXHFZoc+a4UOo6IxHzJKuPluAZ5JoSQOq/YzN2AfG5ixfGSPt+Gw/U01eywIW+hqRcILExknvPzlNu6DWZEDQZ5KGF1+0H0jkMiuJdc/srwmA=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:34.321858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-d8568eaa-d711-4115-af49-7de8b4cfeaa5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:32:34.327580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-d8568eaa-d711-4115-af49-7de8b4cfeaa5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for user 5bb515269d1a45f594f3d297c7fd267e and project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:32:34.357918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d8568eaa-d711-4115-af49-7de8b4cfeaa5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.059657 Sep 17 04:32:34.365581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 228/450] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:32:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:34.825437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:34.826079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:34.834604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:34.834604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:34.834830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:34.835603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:34.836389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:34.836519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:35.030930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "acb9bb23-2e07-483b-b96a-e68d1cb95835", "tag": "b715886e-d53b-4fcc-a366-f48a27e791df"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:35.044731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:35.045293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:35.045987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:35.061823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] Creating event network-changed:b715886e-d53b-4fcc-a366-f48a27e791df for instance acb9bb23-2e07-483b-b96a-e68d1cb95835 on hv-cinder-85799 Sep 17 04:32:35.066982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-0b2f206f-bb18-40ac-96e0-d006c2ea2e27 req-d29ca932-e1e4-4e3a-9c2a-4bead3a6cd6b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041127 Sep 17 04:32:35.067732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 229/451] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:32:35.074631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:35.075404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:35.081714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:35.082235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:35.082646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:35.083483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:35.083784 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:35.084222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:35.085749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:35.087982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:35.095653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:35.097322 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:35.099058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-488ffa37-8fa0-4f00-a98b-1ed5a95e1ed3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.277749 Sep 17 04:32:35.099718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 223/452] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:34 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:35.320008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:35.328107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aacab47f-45dc-4cfa-b177-74bd7320790b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.257880 Sep 17 04:32:35.328672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 230/453] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:35 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:35.801689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1431960219", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8d122af4-d997-47f1-a988-a7aa32c2bb1b"}], "security_groups": [{"name": "tempest-securitygroup--730456468"}], "key_name": "tempest-keypair-1018766987", "user_data": "}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:35.911305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] validate_networks() for [('8d122af4-d997-47f1-a988-a7aa32c2bb1b', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:32:36.123970 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:36.124673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:36.131629 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.132337 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.132846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.133609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.134185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.134688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.270009 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:36.278649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:36.280233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:36.281528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fabb6ee2-4a8d-461a-9f69-c3629e4a674b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.161437 Sep 17 04:32:36.282136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 231/454] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:36 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:36.305543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:32:36.306031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:32:36.306424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:32:36.306838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:32:36.307241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:32:36.316359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.316841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.317329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.318052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.318510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.318931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.335363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:32:36.339346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Getting quotas for user 5bb515269d1a45f594f3d297c7fd267e and project d54f5f66e0eb4be9aad2e6d37e0b6db6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:32:36.350733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:32:36.351404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:36.352015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:36.353016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:32:36.358224 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.358622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.359064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.361827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:36.362296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:36.362742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:36.405926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bd40bc50-a680-4d5b-881e-7bb632567d52 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.608634 Sep 17 04:32:36.406564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 224/455] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:32:35 2022] POST /compute/v2.1/servers => generated 408 bytes in 610 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:32:36.418418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-886deac6-214b-4e01-9af3-0c4831812f5c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:36.419297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-886deac6-214b-4e01-9af3-0c4831812f5c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:36.481154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-886deac6-214b-4e01-9af3-0c4831812f5c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:36.492953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-886deac6-214b-4e01-9af3-0c4831812f5c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1246 microversion: 2.1 time: 0.078753 Sep 17 04:32:36.493705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 225/456] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:36 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1246 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:36.516751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:36.525026 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-42cad143-cbb1-486b-80dc-860716720aba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.177865 Sep 17 04:32:36.526998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 232/457] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:36 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:37.311749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:37.312757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:37.323101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.323916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.324631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.325851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.326604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.327308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.470315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:37.479376 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:37.480814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:37.482002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9319ad68-c437-486e-a0b4-5e30e6270eba tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.177315 Sep 17 04:32:37.482574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 226/458] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:37 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:37.517438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:37.518096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:37.525041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.525566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.526006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.526700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.527155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.527614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.548108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:37.548777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:37.555094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.555580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.556012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.556744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:37.557234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:37.557661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:37.590339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:37.598772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-78413a3f-a19c-4be0-9e31-86d535351d6b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1250 microversion: 2.1 time: 0.085316 Sep 17 04:32:37.599374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 233/459] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:37 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1250 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:37.692423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:37.700582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e5584b2-ff96-459c-a2c9-ce02fc966847 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.156537 Sep 17 04:32:37.701388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 227/460] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:37 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:38.511261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:38.511905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:38.518582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.519178 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.519807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.520781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.521281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.521708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.621525 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:38.622145 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:38.628318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.628811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.629277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.629977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.630558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.630992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.655939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:38.664483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:38.665956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:38.667114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b3d4799c-91e5-4371-bbd7-b5ffdd4e66fe tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.160522 Sep 17 04:32:38.667885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 234/461] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:38 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:38.702807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:38.710923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8b293335-7b34-4e5b-b77b-1d6ae7703093 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1250 microversion: 2.1 time: 0.092998 Sep 17 04:32:38.711645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 228/462] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:38 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1250 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:38.723706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:38.724321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:38.730391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.731090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.731614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.732424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:38.732868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:38.733310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:38.869328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:38.877423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ac8ee420-ef5d-4a78-ae2b-a2587a2dcec4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.158031 Sep 17 04:32:38.877971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 235/463] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:38 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:39.693276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:39.693831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:39.700233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.700713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.701151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:39.701894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.702341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.702756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:39.734706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:39.735254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:39.742003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.742488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.742914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:39.743560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.744256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.744673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:39.808581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:39.817729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f4ccbc6f-5bde-42a7-a4bb-0f0ac527abdc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1298 microversion: 2.1 time: 0.086881 Sep 17 04:32:39.819418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 236/464] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:39 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1298 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:39.837148 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:39.845671 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:39.847122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:39.848648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ddf1cb47-836a-4047-8043-5028c4e94820 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.159426 Sep 17 04:32:39.849483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 229/465] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:39 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:39.900254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:39.900856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:39.908185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.908638 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.909066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:39.909843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:39.910197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:39.910808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:40.046156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:40.054234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c824683f-12a0-43c2-8a8c-1157bcbd0cf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.157861 Sep 17 04:32:40.055042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 237/466] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:39 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:40.841667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:40.842304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:40.848992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:40.849510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:40.849950 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:40.850618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:40.851079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:40.851541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:40.874535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:40.875145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:40.877152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:40.883138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:40.883605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:40.884039 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:40.884776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:40.885230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:40.885658 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:40.915434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:40.923431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c62c55e9-f459-440c-90b9-4eaceefec608 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1304 microversion: 2.1 time: 0.085674 Sep 17 04:32:40.924254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 230/467] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:40 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1304 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:41.079945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:41.080848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:41.091717 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:41.092605 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:41.093134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:41.093969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:41.094488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:41.094977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:41.145676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:41.167698 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:41.177438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:41.178658 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c3035905-e6d2-4c38-b862-3b92c6c5129c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.308932 Sep 17 04:32:41.179227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 238/468] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:40 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:32:41.275683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:41.283495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0c73e6ee-17ca-4ae8-8152-b41313f41b99 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.209580 Sep 17 04:32:41.284017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 231/469] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:41 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:41.820770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:41.826526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:41.831726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:41.832328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:41.832750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:41.846672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] Creating event network-changed:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:32:41.851788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-b835e517-c906-4a9a-b997-2d0a2a59312d req-b9eb4e02-330b-4055-9f35-55c027f16a19 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034894 Sep 17 04:32:41.852499 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 239/470] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:32:41.946952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:41.947565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:41.953526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:41.954257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:41.954663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:41.955541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:41.955969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:41.956571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.080626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "acb9bb23-2e07-483b-b96a-e68d1cb95835", "name": "network-vif-plugged", "status": "completed", "tag": "b715886e-d53b-4fcc-a366-f48a27e791df"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:42.088940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:42.097616 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2c6044b7-62ca-47a2-ad7c-708a2fa8787e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.154174 Sep 17 04:32:42.098342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 232/471] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:41 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:42.098541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:42.099195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:42.099619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.110671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] Creating event network-vif-plugged:b715886e-d53b-4fcc-a366-f48a27e791df for instance acb9bb23-2e07-483b-b96a-e68d1cb95835 on hv-cinder-85799 Sep 17 04:32:42.115477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-15a9daba-a439-4a96-a2a0-1eb7efa26969 req-7e25b443-e4f0-4f22-82eb-7455123631ea service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039289 Sep 17 04:32:42.116304 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 240/472] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:42.204620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:42.205271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:42.212762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:42.213284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:42.213715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.214459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:42.214912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:42.215324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.309297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:42.310668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:42.318000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:42.318860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:42.319516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.320630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:42.321333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:42.321986 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:42.373275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:42.382086 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:42.383688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:42.384935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4c821b68-045d-4c2b-839f-256b9d0b8b27 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.184775 Sep 17 04:32:42.385613 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 233/473] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:42 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:42.662581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:42.672812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d8c1bd84-b1ea-4cb8-a593-fca0cd623cdc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.370627 Sep 17 04:32:42.673571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 241/474] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:42 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:32:43.125214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:43.126238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:43.136679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.137569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.138289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.139414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.140164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.140916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.278731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:43.286694 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6745d7eb-bf23-424d-92ec-e9e8d271b643 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.168250 Sep 17 04:32:43.287237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 234/475] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:43 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:43.410918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:43.411525 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:43.418314 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.418790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.419349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.420167 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.420763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.421369 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.569753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:43.572214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=97,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:43.579458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:43.580889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:43.582100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b7499112-fe8c-46ba-8c98-456498d22f3b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.175210 Sep 17 04:32:43.582593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 242/476] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:43 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:43.697227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:43.697838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:43.704430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.704915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.705383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.706069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:43.706520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:43.706945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:43.846378 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:43.858785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c4945292-11bf-440a-a5ab-2266938b168d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.165872 Sep 17 04:32:43.859611 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 235/477] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:43 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:44.315951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:44.316938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:44.325786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.326261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.326694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:44.327375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.327827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.328241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:44.343465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=16 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:44.461934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:44.469946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4d46018c-de08-407d-b204-fffd10352822 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.160643 Sep 17 04:32:44.470503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 243/478] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:44 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:44.612322 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:44.613222 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:44.621168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.621663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.622097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:44.622789 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.623249 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.623673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:44.756139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:44.764441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:44.765936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:44.767100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bc69616c-2a76-4903-9740-7bfe119aac36 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.161828 Sep 17 04:32:44.767656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 236/479] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:44 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:44.887949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:44.888614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:44.898161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.898815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.899255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:44.899927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:44.900401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:44.900828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:45.044101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:45.052197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-98c3596e-8436-4477-bf30-cc9b1add9c7c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.170917 Sep 17 04:32:45.052741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 244/480] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:44 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:45.493264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:45.493880 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:45.500375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:45.500863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:45.501355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:45.502116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:45.502575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:45.502998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:45.649062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:45.657383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4ed5b8fd-4977-4bc1-bc95-187a2b37a59a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.168060 Sep 17 04:32:45.658120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 237/481] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:45 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:45.793571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:45.794180 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:45.800976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:45.801530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:45.801965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:45.802653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:45.803099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:45.803520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:45.940110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:45.948374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:45.949837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:45.950985 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-29a6e389-4434-4a39-8948-8ea5e574a485 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.161414 Sep 17 04:32:45.951544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 245/482] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:45 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:46.079422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:46.080462 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:46.088874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.089382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.089806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:46.090490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.090940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.091365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:46.224150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:46.232191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9476a04d-ba00-420a-bdf7-0f98d818bf76 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.159389 Sep 17 04:32:46.232746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 238/483] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:46 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:46.682107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:46.682751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:46.689376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.689861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.690317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:46.691009 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.691479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.691912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:46.827890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:46.835929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6170269a-b70b-47c2-8054-fc0831b35cc3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.157799 Sep 17 04:32:46.836495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 246/484] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:46 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:46.980505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:46.981555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:46.984422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:46.985684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=8 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:46.994830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.995319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.995759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:46.996504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:46.996993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:46.997470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:47.130500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:47.143646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:47.146053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:47.147586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-be673816-0b2d-474e-acbb-8e2a8930cc12 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.173698 Sep 17 04:32:47.148462 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 239/485] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:46 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:47.266281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:47.267332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:47.277340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:47.278156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:47.278860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:47.279977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:47.280738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:47.281483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:47.430403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:47.438223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5a5d8507-7d78-40dd-bedc-efbf5a20851a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1365 microversion: 2.1 time: 0.178445 Sep 17 04:32:47.438786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 247/486] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:47 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:47.863196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:47.864270 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:47.871841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:47.872313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:47.872738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:47.873463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:47.873912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:47.874328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.010088 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:48.018120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-37a07fb2-e952-455f-8781-33d6382fcf29 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.161451 Sep 17 04:32:48.018691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 240/487] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:47 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:48.177121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:48.178171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:48.189256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.190063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.190782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.191869 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.192666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.193439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.326777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:48.335703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:48.337152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:48.338348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b4b61e4c-b84f-4939-b60e-802bc25c112d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.167867 Sep 17 04:32:48.338908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 248/488] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:48 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:48.461988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:48.462863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:48.469308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.469948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.470370 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.471229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.471674 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.472454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.606714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:48.614667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ac3b4b72-9a09-4e17-a313-eea996825c8f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1554 microversion: 2.1 time: 0.157406 Sep 17 04:32:48.615282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 241/489] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:48 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1554 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:48.637305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:48.637799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:48.644538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.645139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.645659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.646436 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:48.646958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:48.647481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:48.734637 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b7043014-c14d-4f84-a80a-7b54d9d987b8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.101227 Sep 17 04:32:48.735233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 249/490] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:32:48 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/os-interface => generated 284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:49.040849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:49.041464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:49.047797 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:49.048272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:49.048710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:49.049446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:49.049924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:49.050340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:49.054276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:49.180516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:49.188264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a5fd7a0f-5cc7-42df-b83e-a3156391dbbb tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.152037 Sep 17 04:32:49.188825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 242/491] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:49 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:49.364883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:49.365674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:49.372609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:49.373111 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:49.373537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:49.374268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:49.374716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:49.375134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:49.652270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:49.673587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:49.675027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:49.676226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e0e08ad4-16d4-4dd6-afb3-e6d87b3ae2a7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.316080 Sep 17 04:32:49.676756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 250/492] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:49 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:49.860985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "acb9bb23-2e07-483b-b96a-e68d1cb95835", "tag": "b715886e-d53b-4fcc-a366-f48a27e791df"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:49.870636 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:49.871116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:49.871542 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:49.882860 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] Creating event network-changed:b715886e-d53b-4fcc-a366-f48a27e791df for instance acb9bb23-2e07-483b-b96a-e68d1cb95835 on hv-cinder-85799 Sep 17 04:32:49.888584 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-7bbd1097-5d71-47bc-9f3b-6de572bad261 req-32560bdc-9be8-43cd-bc72-945c761b3793 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031211 Sep 17 04:32:49.889184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 243/493] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:50.211619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:50.212237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:50.218588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:50.219112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:50.219548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:50.220247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:50.220702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:50.221589 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:50.357751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:50.365914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ea336fcc-1b6f-4719-b06a-97eaa1c233ee tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.158298 Sep 17 04:32:50.380982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 251/494] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:50 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:50.703387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:50.704753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:50.711640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:50.712497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:50.712774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:50.713824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:50.714285 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:50.714715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:50.948683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:50.963168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:50.964845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:50.966201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0a56449a-8d88-4f61-b5a4-6d4e720ef636 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.267235 Sep 17 04:32:50.966767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 244/495] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:50 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:51.389296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:51.389296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:51.399357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:51.399853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:51.400280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:51.401004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:51.401470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:51.404262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:51.615080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:51.623245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0fe03aa8-1eca-4dc1-b41a-56cc6f6b2e5a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.238702 Sep 17 04:32:51.623807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 252/496] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:51 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:51.992043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:51.992650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:51.999070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:51.999551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:51.999985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:52.000676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:52.001123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:52.001585 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:52.130914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:52.144486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:52.147021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:52.148747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c08da3d8-9193-471f-89bb-951733fbaddf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.160724 Sep 17 04:32:52.149672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 245/497] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:51 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:52.647380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:52.647979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:52.654892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:52.655379 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:52.655858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:52.656553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:52.657002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:52.657455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:52.803873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:52.811702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4e83b578-49b1-4143-ba47-68a140527607 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.168456 Sep 17 04:32:52.812245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 253/498] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:52 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:52.891604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4dcef58c-6012-43bc-8992-7bd7758bb4b0", "device": "/dev/sdb"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:52.894534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:52.900763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:52.901276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:52.901710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:52.902387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:52.902836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:52.903253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.174730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:53.175428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:53.182033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:53.182514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:53.182944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.183844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:53.184636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:53.185397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.343356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:53.353457 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:53.354905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:53.356086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3993f991-478d-44e1-92d8-ab310ab4b9a3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.185949 Sep 17 04:32:53.356646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 254/499] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:53 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:53.507723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-27575fdb-a97b-4751-816b-af91f9392cd4 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "POST /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.619842 Sep 17 04:32:53.513159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 246/500] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:32:52 2022] POST /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5/os-volume_attachments => generated 194 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:53.513327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "name": "network-vif-plugged", "status": "completed", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:53.543755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:53.544440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:53.545127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.559943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] Creating event network-vif-plugged:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:32:53.567226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-a5b19ecc-4ebd-4c22-b1dc-dc6c3c2e53f4 req-4f356d55-a81c-4179-a881-39ef3dd8322c service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063087 Sep 17 04:32:53.567836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 255/501] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:32:53.834878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:53.835952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:53.843642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:53.844359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:53.844777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.845715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:53.846174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:53.846642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:53.988600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:53.996597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b1472165-a0f4-4bde-b41a-7744d44a760c tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.165904 Sep 17 04:32:53.997187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 247/502] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:53 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:54.387614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:54.388757 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:54.399037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:54.399587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:54.400020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:54.400709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:54.401195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:54.401616 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:54.792546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:54.804480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:54.806643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:54.808414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fa72c2c4-0f76-4491-9fdf-6c00701d46b4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.428952 Sep 17 04:32:54.809149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 256/503] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:54 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:55.023728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:55.024747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:55.035656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:55.036466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:55.037251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:55.038405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:55.039238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:55.039660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:55.176752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:55.184573 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2ec5fe72-e874-44a7-bb05-0dbe193b3508 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.167464 Sep 17 04:32:55.185182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 248/504] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:55 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:55.589774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "name": "network-vif-plugged", "status": "completed", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:55.604728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:55.605231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:55.605662 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:55.616856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] Creating event network-vif-plugged:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:32:55.621803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-4cf70cb1-fc72-4385-9f80-909cc0e18880 req-565ad0d5-f628-46a5-bf9e-81fc1b239a9e service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038535 Sep 17 04:32:55.622378 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 257/505] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:55.841260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:55.842140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:55.845947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:55.855037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:55.855876 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:55.856552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:55.857737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:55.858491 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:55.859194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:56.001029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:56.013654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:56.015834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:56.017627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-18cd8539-a677-46f0-b1f7-83ac49b3d979 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.183067 Sep 17 04:32:56.018440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 249/506] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:55 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:56.212564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:56.213621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:56.224422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:56.225267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:56.225994 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:56.226948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:56.227399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:56.227823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:56.358387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:56.370174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ad90022c-0984-4c85-b363-d879379dbf58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1371 microversion: 2.1 time: 0.164638 Sep 17 04:32:56.370977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 258/507] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:56 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1371 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:57.046667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:57.047635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:57.056431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.057286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.057992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.059130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.059882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.060585 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.191714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:57.200266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:57.201759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:57.202874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e2c2a9c5-f125-458d-a4aa-e48b21960b82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.161778 Sep 17 04:32:57.203612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 250/508] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:57 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:57.394734 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:57.395323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:57.403822 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.404362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.404813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.405726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.406056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.406508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.540875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:57.553443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8b62967f-1669-4512-9b2f-374b5bb7f43d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1565 microversion: 2.1 time: 0.163025 Sep 17 04:32:57.554294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 259/509] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:57 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1565 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:32:57.576629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:57.577053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:57.583203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.583688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.584128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.584815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:57.585320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:57.585781 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:57.665905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6d0a0e79-4dc6-43e3-94fe-32689edd0320 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093214 Sep 17 04:32:57.666507 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 251/510] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:32:57 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:58.228409 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:58.229011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:58.235118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:58.235595 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:58.236022 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:58.236699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:58.237186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:58.237582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:58.452691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:58.461065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:58.462590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:58.463740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9c657906-8488-4f45-9f3e-f89593ca0c8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1740 microversion: 2.20 time: 0.239378 Sep 17 04:32:58.464299 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 260/511] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:58 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1740 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:32:58.729204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:58.739241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:58.739723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:58.740143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:58.751392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] Creating event network-changed:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:32:58.756378 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-ce9e1372-24b3-414e-a356-5d2d482ecae3 req-465f5d69-23b5-4384-893e-defd1477c26a service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031444 Sep 17 04:32:58.756979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 252/512] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:32:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:32:59.483505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:32:59.487922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:59.490319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:59.491106 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:59.499560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.500336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.500933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.502613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.502743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.503466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.504066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.504507 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.505292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.505412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.508261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.508690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.539163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:32:59.604371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0bf363dc-82cd-4438-a77a-3cb5d1e2b1d3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action" status: 202 len: 0 microversion: 2.1 time: 0.128611 Sep 17 04:32:59.605703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 261/513] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:32:59 2022] POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:32:59.614310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:32:59.615299 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:32:59.622961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.623851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.624465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.625376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:32:59.625992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:32:59.626468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:32:59.695561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:59.715680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:59.718100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:59.719527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5a875451-64a0-45d4-b4cf-4b3751b0612a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.234503 Sep 17 04:32:59.728777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 253/514] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:32:59 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:32:59.842592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:32:59.852180 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc53cfd2-d6ff-42d1-913d-d06f44f6bb4e tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1752 microversion: 2.1 time: 0.242434 Sep 17 04:32:59.868359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 262/515] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:32:59 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1752 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:00.748960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:00.749620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:00.758145 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:00.759419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:00.760336 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:00.761451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:00.762157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:00.762813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:00.875990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:00.876597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:00.883206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:00.883907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:00.884336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:00.885032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:00.885506 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:00.885925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:00.927513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:00.940059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:00.941903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:00.943095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-81f385c8-87e7-436b-8493-2af04405607b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.198119 Sep 17 04:33:00.943651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 254/516] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:00 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:01.046013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:01.055423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5810adcf-a353-4b9e-a925-910af1ba9a19 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.183272 Sep 17 04:33:01.056005 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 263/517] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:00 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:01.969000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:01.969675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:01.976550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:01.977040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:01.977520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:01.978201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:01.978648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:01.979068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:02.079085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:02.080332 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:02.088381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:02.089063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:02.090770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:02.091710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:02.092564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:02.093389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:02.140662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:02.153255 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:02.155005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:02.156712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c02a685e-d25f-4866-85f8-0de6bb67cf8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.191664 Sep 17 04:33:02.157810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 255/518] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:01 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:02.250319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:02.259056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8067cfeb-6c5b-424e-bed0-c5d97d4d612b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.183995 Sep 17 04:33:02.259611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 264/519] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:02 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:03.187626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:03.188233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:03.195240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:03.195709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:03.196300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:03.197046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:03.197524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:03.198065 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:03.282737 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:03.284194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:03.292886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:03.293781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:03.294293 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:03.295446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:03.296144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:03.296768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:03.430644 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:03.445393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:03.450108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:03.452537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2f53e1e7-9d38-4364-97d8-4ff99e730392 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.268729 Sep 17 04:33:03.472999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 256/520] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:03 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:03.507679 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:03.516187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-90738855-49ab-48ff-b068-219cf8085451 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.237858 Sep 17 04:33:03.516745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 265/521] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:03 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:04.483393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:04.484833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:04.492161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:04.492716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:04.493248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:04.494004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:04.494484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:04.494906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:04.546153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:04.547700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:04.557389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:04.559562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:04.560200 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:04.561320 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:04.561847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:04.562275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:04.677236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:04.695710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:04.697189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:04.699381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-691621b9-e869-411a-b389-b97578de8c16 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.219925 Sep 17 04:33:04.700584 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 257/522] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:04 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:04.786630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:04.795736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aaba9e26-72fa-4cd8-a1d0-72c610a91663 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.258238 Sep 17 04:33:04.796937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 266/523] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:04 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:05.725108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:05.725708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:05.732407 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:05.732861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:05.733576 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:05.734631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:05.735060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:05.735663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:05.820503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:05.823324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:05.835253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:05.835807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:05.836246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:05.836951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:05.837610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:05.838199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:05.881905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:05.890612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:05.892823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:05.894137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a9f2dce1-1afc-4953-8b5a-398cf0802dee tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.173372 Sep 17 04:33:05.894949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 258/524] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:05 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:06.028021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:06.036768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-36c8bdac-25d0-4cf9-8652-97ac487feeac tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.220696 Sep 17 04:33:06.037380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 267/525] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:05 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:06.922076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:06.922681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:06.930226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:06.930949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:06.931551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:06.932241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:06.932685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:06.933164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:06.935945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:07.060142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:07.060817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:07.068077 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:07.068905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:07.070280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:07.071109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:07.071984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:07.072652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:07.088156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:07.098394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:07.101702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:07.103935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9a69616c-132c-4bd6-9b39-4b9534e1b1d3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.185446 Sep 17 04:33:07.117107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 259/526] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:06 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:07.243120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:07.252477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-299fd61e-d3b1-40b7-8ba6-11a37d8aa497 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.196261 Sep 17 04:33:07.253046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 268/527] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:07 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:08.129395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:08.130731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:08.139474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:08.139974 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:08.140563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:08.141284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:08.141945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:08.142559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:08.282742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:08.284166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:08.291807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:08.292666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:08.293351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:08.294294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:08.294951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:08.295572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:08.310511 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:08.320031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:08.323929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:08.325159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8364c7b8-e0b7-4d18-9938-051e5b9d9f14 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.199699 Sep 17 04:33:08.326998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 260/528] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:08 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:08.463818 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:08.472598 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2005c220-9062-49ee-a8f3-cf57e3291bcd tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.196687 Sep 17 04:33:08.473376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 269/529] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:08 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:09.354509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:09.355451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:09.363956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:09.364750 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:09.365499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:09.366384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:09.367104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:09.367531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:09.496029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:09.496641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:09.504969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:09.505596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:09.506040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:09.506931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:09.507580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:09.508810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:09.530834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:09.544756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:09.546719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:09.549814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3a4f4191-f2ef-43f5-aa17-8e1d1e80d73b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.201298 Sep 17 04:33:09.550602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 261/530] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:09 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:09.680330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:09.691285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d4ac549f-b0bb-4777-86fd-75010d769f45 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.199203 Sep 17 04:33:09.692437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 270/531] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:09 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:10.583354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:10.584239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:10.595394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:10.596201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:10.597383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:10.598518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:10.599272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:10.599982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:10.715746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:10.716331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:10.723984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:10.725026 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:10.726190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:10.726920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:10.727464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:10.727942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:10.740484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:10.748569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:10.750084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:10.751232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-62091da2-930a-4c5c-9579-3a7cf561554e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.174691 Sep 17 04:33:10.751768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 262/532] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:10 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:10.874859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:10.887444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0566f7b8-c051-4570-b522-215e85e5458d tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.175481 Sep 17 04:33:10.888247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 271/533] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:10 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:11.782416 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:11.783426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:11.794505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:11.795313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:11.796046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:11.797288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:11.798055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:11.798762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:11.911856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:11.912434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:11.914943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=66 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:11.920699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:11.921203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:11.921634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:11.922319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:11.922747 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:11.923164 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:11.929852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:11.938273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:11.939707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:11.940864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7e426578-e0c2-480b-a64d-e2c76c4ede45 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.165161 Sep 17 04:33:11.941771 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 263/534] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:11 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:12.065390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:12.074069 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3838f5f6-b0b3-4dbf-b6bb-fa9919d58edb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.166164 Sep 17 04:33:12.074685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 272/535] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:11 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:12.970171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:12.971071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:12.980759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:12.981289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:12.981730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:12.982426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:12.982886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:12.983317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:13.098069 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:13.098667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:13.105459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:13.105965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:13.106402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:13.107066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:13.107532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:13.107952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:13.119999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:13.131656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:13.133975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:13.135704 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d299ac35-98d1-4c68-804c-922410fba0c3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.172095 Sep 17 04:33:13.136519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 264/536] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:12 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:13.263064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:13.271500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8e39f16a-4bee-4c4a-87f3-8b54c6eeef63 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.177672 Sep 17 04:33:13.272042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 273/537] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:13 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:14.173179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:14.174212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:14.185584 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:14.186394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:14.187111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:14.188305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:14.189070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:14.189822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:14.294799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:14.295406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:14.302389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:14.302884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:14.303323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:14.304026 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:14.304483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:14.304909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:14.321380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:14.329653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:14.331094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:14.332264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9993ca63-e6ba-43a6-b495-c2e7b808a683 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1735 microversion: 2.20 time: 0.165828 Sep 17 04:33:14.332861 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 265/538] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:14 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:14.463098 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:14.472082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8fba3301-5e63-4876-a0be-45ff8135f29b tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.180825 Sep 17 04:33:14.472883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 274/539] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:14 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:15.357110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:15.357831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:15.364485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:15.365057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:15.365540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:15.366209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:15.366662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:15.367087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:15.496152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:15.496727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:15.503301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:15.503748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:15.504173 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:15.504878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:15.505390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:15.505810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:15.545029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:15.554060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:15.555534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:15.556711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cc7e7057-2be1-4d74-b894-1ed3a1d0472c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.203602 Sep 17 04:33:15.557333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 266/540] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:15 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:15.656244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:15.665281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5b949596-d1b4-463b-bd4c-a1c605727275 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.173153 Sep 17 04:33:15.665866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 275/541] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:15 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:16.586313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:16.587387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:16.598781 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:16.599638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:16.600361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:16.601549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:16.602361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:16.602944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:16.688091 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:16.688780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:16.695581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:16.696065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:16.696638 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:16.697365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:16.697813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:16.698223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:16.740308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:16.748855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:16.750340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:16.751529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c4bf07e5-37dc-4921-92bb-b91d9cd1350f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.171902 Sep 17 04:33:16.752092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 267/542] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:16 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:16.844664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:16.847265 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:16.855489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-90b188b7-b4a0-4829-b13d-da48ed59d7e3 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.171380 Sep 17 04:33:16.856034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 276/543] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:16 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:17.781031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:17.782099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:17.793203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:17.794049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:17.794776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:17.795911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:17.796667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:17.797445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:17.884267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:17.884868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:17.892643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:17.893139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:17.893589 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:17.894271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:17.894718 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:17.895123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:17.938394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:17.947014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:17.948466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:17.949659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9d69a810-7a3e-4f8d-b1d5-e30b775cf844 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.175446 Sep 17 04:33:17.950216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 268/544] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:17 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:18.036073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:18.044961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-92233914-ccde-48ce-b9c9-8606e71684bb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.165015 Sep 17 04:33:18.045574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 277/545] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:17 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:18.978994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:18.979665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:18.988052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:18.988662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:18.989259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:18.990123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:18.990693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:18.991227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:19.068134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:19.068739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:19.075496 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:19.076008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:19.076426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:19.077117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:19.077602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:19.078031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:19.138091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:19.146741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:19.148203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:19.149427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-64f7c12e-630c-4477-90b0-b66be2b2c372 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.176908 Sep 17 04:33:19.150016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 269/546] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:18 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:19.229484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:19.242238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a2d334d1-507c-4e73-b252-b50788b11d99 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.177849 Sep 17 04:33:19.243191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 278/547] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:19 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:20.174023 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:20.174656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:20.181870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:20.182362 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:20.182798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:20.183497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:20.183948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:20.184377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:20.266955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:20.267608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:20.275269 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:20.275827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:20.276422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:20.277462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:20.277995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:20.278618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:20.376488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:20.388518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:20.391472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:20.393650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6b8a1762-a181-4e25-bf61-76b073f1e479 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.223610 Sep 17 04:33:20.396151 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 270/548] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:20 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:20.404120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "name": "network-vif-unplugged", "status": "completed", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:33:20.416003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:20.417446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:20.418057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:20.435563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] Creating event network-vif-unplugged:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:33:20.440651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-d7938e3d-8194-4f62-8c8e-ddd2db99e2fe req-717d71af-fe0e-4c77-bfb3-d8cb905656d5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042683 Sep 17 04:33:20.441320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 271/549] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:33:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:33:20.456840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:20.465407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-83ad1806-ef5b-499f-9c74-5adec12380f1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.203426 Sep 17 04:33:20.465974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 279/550] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:20 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:21.424392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:21.425795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:21.434840 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:21.435329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:21.435763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:21.436486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:21.436934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:21.437394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:21.489327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:21.489932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:21.496561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:21.497014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:21.497502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:21.498190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:21.498647 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:21.499070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:21.570035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:21.578242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:21.579673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:21.580809 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7fd6903c-bcbd-4af7-b999-525126a2895a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1717 microversion: 2.20 time: 0.164674 Sep 17 04:33:21.581444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 272/551] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:21 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1717 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:21.761518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:21.770199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-55bb4a1d-7067-4e44-b6f4-0e0b0260dccb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.284859 Sep 17 04:33:21.770779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 280/552] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:21 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:22.607413 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:22.608067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:22.614829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.615602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.616072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.616751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.617363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.617812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.801614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:22.808462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:22.821362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.822084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.823361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.825855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.826798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.827379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:22.828935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.838949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:22.840548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:22.841955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3e8f0be3-1a33-47bd-b906-cbb7c72fceb9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1720 microversion: 2.20 time: 0.238569 Sep 17 04:33:22.842505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 273/553] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:22 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1720 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:22.867200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0d8ba18e-24b7-457e-8524-26fd4cb189bf"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:33:22.869749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:22.876239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.876727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.877214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.877944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:22.878392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:22.878806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:22.979834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:22.988456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-be5f5e25-d1fa-4664-91bd-c8bcd42e59c9 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.192492 Sep 17 04:33:22.989263 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 281/554] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:22 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:23.080572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:23.182801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d631ef51-3fda-4e92-bdfe-51ad3cbdfc56 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.319654 Sep 17 04:33:23.183466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 274/555] 10.222.0.60 () {62 vars in 1364 bytes} [Sat Sep 17 04:33:22 2022] POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 188 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:33:23.324849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:33:23.325564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:23.332003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:23.332485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:23.332916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:23.333635 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:23.334083 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:23.334503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:23.377185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120626) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 17 04:33:23.454482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-819fdd33-ee66-496d-b73b-a10db9981b09 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/action" status: 202 len: 0 microversion: 2.20 time: 0.133630 Sep 17 04:33:23.455102 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 282/556] 10.222.0.60 () {62 vars in 1319 bytes} [Sat Sep 17 04:33:23 2022] POST /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:33:23.463629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:23.464544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:23.471252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:23.472098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:23.472526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:23.473245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:23.473678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:23.474113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:23.615427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:23.624661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:23.626129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:23.627282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-43461ab9-ea07-46e9-9e74-22ac04253cb4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1746 microversion: 2.20 time: 0.167778 Sep 17 04:33:23.628010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 275/557] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:23 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1746 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:24.013747 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:24.014519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:24.020876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.021399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.021835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:24.022532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.022957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.023385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:24.172403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:24.181135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0c7f8aee-117d-418a-b5e6-e0a928190c7f tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.171693 Sep 17 04:33:24.181795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 283/558] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:24 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:24.652021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:24.652644 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:24.659423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.660083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.660513 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:24.662699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.663169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.664121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:24.797893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:24.806876 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:24.808298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:24.809480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5128d63e-70f3-4928-bbd4-5ba33eb8ff67 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1748 microversion: 2.20 time: 0.161366 Sep 17 04:33:24.810038 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 276/559] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:24 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1748 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:24.988267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:24.988867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:24.995110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.995600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.996032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:24.996730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:24.997217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:24.997642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:25.127965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lazy-loading 'services' on Instance uuid 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:33:25.204749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:25.205466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:25.211617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:25.212229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:25.212643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:25.213369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:25.213826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:25.214254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:25.250700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-030a4ba4-aa4b-4cad-b86c-e01fcc9e3eb0 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "DELETE /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5/os-volume_attachments/4dcef58c-6012-43bc-8992-7bd7758bb4b0" status: 202 len: 0 microversion: 2.1 time: 0.266273 Sep 17 04:33:25.251272 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 284/560] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:33:24 2022] DELETE /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5/os-volume_attachments/4dcef58c-6012-43bc-8992-7bd7758bb4b0 => generated 0 bytes in 268 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:33:25.368276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:25.380014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4e7468aa-8354-4f39-a0c0-f8fcebf302ea tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1757 microversion: 2.1 time: 0.178773 Sep 17 04:33:25.380886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 277/561] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:25 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1757 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:25.839442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:25.840329 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:25.849568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:25.850197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:25.850761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:25.851659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:25.852248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:25.852785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:26.001511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:26.014876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:26.017123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:26.018895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a312ee52-2b62-43dc-adee-f1b3f0b2838c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1748 microversion: 2.20 time: 0.186273 Sep 17 04:33:26.019662 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 285/562] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:25 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1748 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:26.407715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:26.408362 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:26.411773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=58 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:26.417857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:26.418338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:26.418777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:26.419492 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:26.419946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:26.420362 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:26.567837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:26.581424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-70d05336-96c2-4e23-ab81-e7717a212de1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1760 microversion: 2.1 time: 0.179719 Sep 17 04:33:26.582352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 278/563] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:26 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1760 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:27.046373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:27.047037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:27.053714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.054198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.054633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.055363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.055797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.056214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.188264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:27.197125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:27.198547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:27.199708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0bb0ce86-6c8f-4f19-b289-07c01a174736 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.157608 Sep 17 04:33:27.200232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 286/564] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:27 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:27.446148 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:27.446640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:27.453700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.454192 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.454634 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.455323 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.455779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.456201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.486563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:33:27.572077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e8b4696d-a551-497d-aa63-60206b4746f7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "DELETE /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 204 len: 0 microversion: 2.1 time: 0.129439 Sep 17 04:33:27.572946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 279/565] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:33:27 2022] DELETE /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:33:27.581473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:27.582082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:27.588597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.589128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.589533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.590218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:27.590664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:27.591083 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:27.728688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:27.736973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-934a5834-f3ef-4483-9e39-f05ae81bab73 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1513 microversion: 2.1 time: 0.159617 Sep 17 04:33:27.737556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 287/566] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:27 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1513 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:28.230005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:28.231147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:28.242993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:28.243772 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:28.244206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:28.244889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:28.245893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:28.246320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:28.377234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:28.386004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:28.387431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:28.388587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-57dc6e19-3e80-4980-938d-009e3c443e87 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.165847 Sep 17 04:33:28.389131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 280/567] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:28 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:28.764759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:28.765809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:28.776070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:28.776556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:28.776990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:28.777719 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:28.778170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:28.778632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:28.910353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:28.923215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7b934ef5-b0bf-4df1-b430-865cf5920218 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1513 microversion: 2.1 time: 0.164850 Sep 17 04:33:28.923706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 288/568] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:28 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1513 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:29.413782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:29.414398 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:29.421641 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:29.422133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:29.422581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:29.423333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:29.423796 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:29.424258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:29.559106 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:29.568690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:29.570150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:29.571305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ef336ec3-9f7c-4d09-aac9-a64218844cce tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.161531 Sep 17 04:33:29.571856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 281/569] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:29 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:29.954357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:29.955145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:29.964255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:29.964894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:29.965527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:29.966450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:29.967053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:29.967625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:30.110161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:30.118377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-486971a9-d176-4b0f-aa8f-77148c2acb3b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1513 microversion: 2.1 time: 0.170190 Sep 17 04:33:30.118952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 289/570] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:29 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1513 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:30.603230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:30.603799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:30.614934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:30.617958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:30.619919 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:30.620866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:30.624668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:30.625888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:30.932741 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:30.943731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:30.945617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:30.949324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1c3895d0-0d53-4091-a631-425b196006da tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.348674 Sep 17 04:33:30.954396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 282/571] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:30 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:30.966668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "564a5f73-27b3-43f1-9c5f-3c69ecc186a5", "name": "network-vif-deleted", "tag": "9b74c853-353e-4c20-a37a-ebb673727be5"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:33:30.978990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:30.979464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:30.979889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:30.991517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] Creating event network-vif-deleted:9b74c853-353e-4c20-a37a-ebb673727be5 for instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 on hv-cinder-85799 Sep 17 04:33:30.998251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-5c86d00a-6cc0-43f8-b2b2-bb5e7ff7a06a req-8d990233-00be-4e1a-934a-0bee56e7c7e1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036769 Sep 17 04:33:30.998919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 290/572] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:33:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:33:31.143519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:31.144240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:31.151193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:31.151684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:31.152116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:31.152822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:31.153316 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:31.153739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:31.217931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:31.226358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f25cc009-b7ed-475d-b776-b4c12e27b141 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.088348 Sep 17 04:33:31.228214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 283/573] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:31 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:31.982357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:31.983511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:31.993942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:31.994735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:31.995535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:31.996225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:31.996667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:31.997057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:32.131049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:32.140149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:32.141924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:32.143094 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8151c5d2-fc2a-44c9-b196-f8efd2bba927 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.169481 Sep 17 04:33:32.143623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 291/574] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:31 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:32.252617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:32.253224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:32.260352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:32.260962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:32.261453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:32.262159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:32.262600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:32.263051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:32.332837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:32.341026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e16404bc-7a48-41df-b2bb-4d79add6ac60 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.094459 Sep 17 04:33:32.341818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 284/575] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:32 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:33.168670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:33.169399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:33.176422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:33.176944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:33.177405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:33.178105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:33.178554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:33.178977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:33.194798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:33.328516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:33.337555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:33.338991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:33.340158 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0aaef387-e740-4eae-b451-1d787e8f8edb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.175574 Sep 17 04:33:33.340992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 292/576] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:33 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:33.367499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:33.368161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:33.374897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:33.375381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:33.375819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:33.376514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:33.376964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:33.377411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:33.440476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:33.448358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d7e6b777-f496-4bc6-b1d8-40d0035185fa tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.086776 Sep 17 04:33:33.448933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 285/577] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:33 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:34.366360 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:34.367027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:34.373843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:34.374328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:34.374776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:34.375463 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:34.375913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:34.376335 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:34.471920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:34.473221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:34.480839 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:34.481631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:34.482067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:34.482764 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:34.483422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:34.483845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:34.515710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:34.525567 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:34.527033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:34.528197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d56e7051-4aa8-4b2f-81f5-d80daa78a97c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.165848 Sep 17 04:33:34.529117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 293/578] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:34 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:34.558191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:34.566305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-80271d52-58cb-4762-8ab0-c5df0b47759a tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.098622 Sep 17 04:33:34.567070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 286/579] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:34 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:33:35.553753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:35.554399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:35.561590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:35.562049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:35.562492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:35.563201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:35.563657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:35.564084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:35.590700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:35.591351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:35.599385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:35.600087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:35.600730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:35.601495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:35.602051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:35.602529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:35.674044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:35.682344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ec3c62af-e45e-474d-901f-eed9ae9b8907 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.096485 Sep 17 04:33:35.683508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 287/580] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:35 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:35.714583 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:35.724267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:35.725796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:35.726958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-595ba579-8d6d-4bd2-814e-0eac2bfb3f63 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.177346 Sep 17 04:33:35.727538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 294/581] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:35 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:36.705495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:36.706777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:36.713497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:36.713967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:36.714395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:36.715276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:36.715724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:36.716176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:36.752191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:36.752821 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:36.759742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:36.760244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:36.760713 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:36.761427 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:36.761875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:36.762294 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:36.780556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:36.783098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:UPDATE=4,nova_cell1:INSERT=2,nova_cell1:SELECT=94 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:36.793105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a51cc7c0-8475-4ba3-ad49-0cff716dff3c tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.091371 Sep 17 04:33:36.793677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 288/582] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:36 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:36.902756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:36.912023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:36.919524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:36.920867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9b7b232d-3350-4f7c-9b4f-315f475ff6b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.172595 Sep 17 04:33:36.921483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 295/583] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:36 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:33:37.817787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:37.818395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:37.826724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:37.827301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:37.827743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:37.828653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:37.829139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:37.829569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:37.895156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:37.903702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5d7f0989-3718-4be7-9841-5aa59c240d95 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.091025 Sep 17 04:33:37.908955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 289/584] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:37 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:37.947799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:37.948467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:37.955938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:37.956680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:37.957256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:37.957803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:37.958262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:37.958699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:38.093060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:38.103074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:38.104534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:38.105765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b0f3c669-49cf-488e-bfef-bf4ee4e22cd4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.161977 Sep 17 04:33:38.106300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 296/585] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:37 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:38.926663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:38.927379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:38.934699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:38.935183 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:38.935615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:38.936349 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:38.936800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:38.937245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:39.002385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:39.010429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-650a4721-b06d-4fe5-9269-9d3f2c340bdb tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.088003 Sep 17 04:33:39.011003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 290/586] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:38 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:39.131386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:39.132108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:39.139368 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:39.139855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:39.140321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:39.141023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:39.141523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:39.141939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:39.285733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:39.295049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:39.296654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:39.297998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-28c4a6ab-f03f-4746-91b7-c0fbaed0d941 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.170547 Sep 17 04:33:39.298773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 297/587] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:39 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:40.034758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:40.035381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:40.042365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:40.042851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:40.043284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:40.043996 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:40.044445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:40.044864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:40.110163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:40.118923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f15f39f2-a5a9-4e57-b428-21a7393d6ba8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.088651 Sep 17 04:33:40.119493 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 291/588] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:40 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:40.323825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:40.324513 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:40.339169 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:40.339659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:40.340091 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:40.340790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:40.341344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:40.341777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:40.476447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:40.486095 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:40.487815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:40.488998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc77b22c-a2eb-40e3-b2ed-c5717f43f13b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.169540 Sep 17 04:33:40.489622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 298/589] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:40 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:41.142575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:41.143326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:41.150159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:41.150681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:41.151112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:41.151843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:41.152287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:41.152699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:41.220048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:41.228645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-478a90ab-7fb8-4899-8415-495ac23fb1d3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.089972 Sep 17 04:33:41.229263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 292/590] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:41 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:41.515564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:41.516385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:41.523136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:41.523622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:41.524186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:41.524883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:41.525435 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:41.525828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:41.658633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:41.668968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:41.670461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:41.671641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5dd67e82-009b-4dac-be78-e17973a44158 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.160544 Sep 17 04:33:41.672190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 299/591] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:41 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:42.253720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:42.254340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:42.260471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:42.260943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:42.261406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:42.262080 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:42.262544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:42.262961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:42.325873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:42.334565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ece563ae-127c-4408-b276-abce78fe97ee tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.086378 Sep 17 04:33:42.335187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 293/592] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:42 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:42.696211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:42.696936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:42.699897 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:42.705768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:42.706182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:42.706613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:42.707347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:42.707797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:42.708222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:42.849315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:42.858957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:42.860515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:42.861785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a98a92df-59a1-4b24-8b31-709231838384 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.169595 Sep 17 04:33:42.862324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 300/593] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:42 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:43.362528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:43.363552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:43.374501 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:43.375350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:43.376064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:43.376759 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:43.377250 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:43.377680 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:43.440557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:43.448592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1a409d4d-6bf8-4a77-9afe-443713fc0a8d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.092793 Sep 17 04:33:43.449157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 294/594] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:43 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:43.891192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:43.892004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:43.899001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:43.899489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:43.899923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:43.900619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:43.901184 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:43.901614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:44.042430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:44.054468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:44.056723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:44.058540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-79685a88-dbd2-4bdf-9ed7-e24f4cc92382 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.173770 Sep 17 04:33:44.059351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 301/595] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:43 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:44.474214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:44.474922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:44.483135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:44.483731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:44.484266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:44.485142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:44.485709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:44.486222 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:44.558993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:44.567548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-36483529-a6d5-4cd5-9919-780e1a66b0f6 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.098693 Sep 17 04:33:44.568142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 295/596] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:44 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:45.089532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:45.090508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:45.102033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:45.102836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:45.103571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:45.104847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:45.105686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:45.106403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:45.252493 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:45.265832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:45.267982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:45.269140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b3916054-0e41-4918-8e2d-9e2d25ff1ed8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.186283 Sep 17 04:33:45.269724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 302/597] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:45 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:45.594536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:45.595565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:45.604001 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:45.604531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:45.605014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:45.605826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:45.606346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:45.606811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:45.675055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:45.683210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-244232b5-715e-4628-9fd8-6bd2a22cbdd5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1471 microversion: 2.1 time: 0.095296 Sep 17 04:33:45.683700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 296/598] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:45 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1471 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:46.298379 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:46.299420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:46.310465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:46.311312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:46.312044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:46.313177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:46.313979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:46.314841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:46.457350 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:46.466067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:46.467543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:46.468741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a8315232-3418-4a79-bf4f-7ac6df25510a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.177040 Sep 17 04:33:46.469344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 303/599] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:46 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:46.707599 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:46.708178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:46.715463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:46.715987 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:46.716440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:46.717180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:46.717639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:46.718061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:46.789803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:46.797805 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-966072ac-6fab-4485-a6f9-e0e5543feef7 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1303 microversion: 2.1 time: 0.094776 Sep 17 04:33:46.798368 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 297/600] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:46 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1303 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:47.494706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:47.495219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:47.502264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:47.502752 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:47.503183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:47.503884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:47.504327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:47.504750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:47.645495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:47.654376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:47.655844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:47.657006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9febb441-d164-485d-82c0-3565ec32fd69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.166400 Sep 17 04:33:47.661729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 304/601] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:47 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:47.821320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:47.821947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:47.838617 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:47.839111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:47.839546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:47.840266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:47.840718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:47.841164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:47.873868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:47.874356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:47.874800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:47.875248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lazy-loading 'fault' on Instance uuid 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:33:47.926911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:47.935226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1d3b8856-50ec-4b6c-80b1-bacc1673f269 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 200 len: 1308 microversion: 2.1 time: 0.117915 Sep 17 04:33:47.935751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 298/602] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:47 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 1308 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:33:48.684991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:48.685851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:48.693837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:48.694360 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:48.694796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:48.695469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:48.695948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:48.696373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:48.834878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:48.843615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:48.845034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:48.846573 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5663c1d8-983c-4d32-91e1-8d11984d9b32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.166918 Sep 17 04:33:48.847142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 305/603] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:48 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:48.958592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:48.959160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 564a5f73-27b3-43f1-9c5f-3c69ecc186a5] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:48.966221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:48.966711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:48.967140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:48.967833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:48.968289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:48.968715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:48.982720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] HTTP exception thrown: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Request method failure captured: Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 HTTP/1.1 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU3CXnLmMd8_kli1tUzeSDDVRswtO5MCz6Xc0RVfUEVubONjZE2sBZmneO8bZkzgy1SIqmSE9bERdE1YbmpLalU1r1MeaanJei2Du8KMVRt2bELC2SPDp3akF9V05hImtp2emS1dHlLCMpJYuKu_KrbAYsl3pLQj8Nf5IR0xg6-QQGI Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 9d47a53afdbf404a829c119992fcb643 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:33:48.984412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_9d47a53afdbf404a829c119992fcb643"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 9d47a53afdbf404a829c119992fcb643 Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesSnapshotTestJSON-1455811953-project Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 166d99888375478e9cd8bf8dcb9d4190 Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesSnapshotTestJSON-1455811953-project Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. Sep 17 04:33:48.986092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '564a5f73-27b3-43f1-9c5f-3c69ecc186a5'} Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:33:48.987058 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. Sep 17 04:33:48.988563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:33:48.990679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Returning 404 to user: Instance 564a5f73-27b3-43f1-9c5f-3c69ecc186a5 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:33:48.990679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6d8487a0-0ea8-41ba-97db-c515d0d89b8b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5" status: 404 len: 111 microversion: 2.1 time: 0.031388 Sep 17 04:33:48.990679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 299/604] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:33:48 2022] GET /compute/v2.1/servers/564a5f73-27b3-43f1-9c5f-3c69ecc186a5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:33:49.880773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:49.881771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:49.892975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:49.893486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:49.893912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:49.894589 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:49.895039 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:49.895456 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:50.034592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:50.043508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:50.044934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:50.046117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-413b6b7c-d137-440c-a7c9-94bc2c75178b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.171806 Sep 17 04:33:50.046684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 306/605] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:49 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:51.075499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:51.076510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:51.085053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:51.085573 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:51.086005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:51.086691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:51.087143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:51.087565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:51.222260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:51.230718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:51.232181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:51.233355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cafc4f08-4c02-4036-b567-e45b78ee8a44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.164554 Sep 17 04:33:51.233904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 300/606] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:51 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:52.263580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:52.264580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:52.275369 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:52.276171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:52.276896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:52.278083 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:52.278842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:52.279546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:52.421116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:52.430313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:52.431745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:52.432907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-eaaadbbd-4d2f-4b1e-93be-2d1323db4dcd tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.175862 Sep 17 04:33:52.433501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 307/607] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:52 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:53.458825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:53.459444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:53.471268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:53.471895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:53.472780 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:53.474742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:53.478067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:53.478067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:53.635490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:53.645050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:53.647359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:53.649143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ade4d950-52d6-484e-bc72-0cdc59dade30 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.194559 Sep 17 04:33:53.650007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 301/608] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:53 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:54.678930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:54.679800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:54.689710 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:54.690501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:54.691123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:54.692195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:54.692641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:54.693061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:54.824189 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:54.832997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:54.834345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:54.835523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-983e3f05-1cae-4624-953c-e61fa8e4218f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.162252 Sep 17 04:33:54.836075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 308/609] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:54 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:55.861573 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:55.862168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:55.869481 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:55.869964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:55.870404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:55.871081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:55.871534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:55.871954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:56.003091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:56.012396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:56.013837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:56.014991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-07fda0f4-fc1b-4869-b73e-924d7a4c7e89 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.157520 Sep 17 04:33:56.015545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 302/610] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:55 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:57.043922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:57.044949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:57.054121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:57.054598 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:57.055023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:57.055706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:57.056151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:57.056568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:57.059122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:57.191940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:57.205161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:57.207431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:57.208577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8fd22657-7cec-4681-be78-3c14b9242361 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.171408 Sep 17 04:33:57.209129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 309/611] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:57 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:58.238478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:58.239335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:58.243100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:33:58.252276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:58.253143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:58.253869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:58.254762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:58.255220 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:58.255647 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:58.390010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:58.403744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:58.406033 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:58.407807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6de1e96e-8fdd-4e7c-93f6-b564d798c14f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.176012 Sep 17 04:33:58.408562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 303/612] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:58 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:33:59.439026 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:33:59.440040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:33:59.451342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:59.452128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:59.452886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:59.454072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:33:59.454824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:33:59.455522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:33:59.593174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:59.605815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:59.608007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:33:59.609614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-55095ba4-b7ec-497f-bac5-67e8c7fc9bc5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.177602 Sep 17 04:33:59.610403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 310/613] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:33:59 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:00.638872 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:00.639933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:00.651186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:00.651997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:00.652723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:00.653964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:00.654738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:00.655499 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:00.790700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:00.803141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:00.804664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:00.805863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a0c09ae4-7380-499a-8916-f0c9de7ba03f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.173612 Sep 17 04:34:00.806420 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 304/614] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:00 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:01.057137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1075883605", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "networks": [{"uuid": "bb95167b-27c5-4a3a-8d9b-1b9930df9108"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:01.112687 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] validate_networks() for [('bb95167b-27c5-4a3a-8d9b-1b9930df9108', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:34:01.522365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:34:01.522365 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:34:01.522939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:34:01.523346 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:34:01.523755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:34:01.532733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:01.533253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:01.533695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:01.534414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:01.534867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:01.535290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:01.553625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Getting quotas for project 9d47a53afdbf404a829c119992fcb643. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:34:01.558006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Getting quotas for user 166d99888375478e9cd8bf8dcb9d4190 and project 9d47a53afdbf404a829c119992fcb643. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:34:01.570533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:34:01.572883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:34:01.625957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-15fb012d-797c-4e4b-8e75-d5b801147a42 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.572716 Sep 17 04:34:01.626642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 311/615] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:34:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 574 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:34:01.638448 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7abb7936-79a1-4188-915f-fbed55a874a9 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:01.639084 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7abb7936-79a1-4188-915f-fbed55a874a9 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:01.694951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7abb7936-79a1-4188-915f-fbed55a874a9 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:01.702193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7abb7936-79a1-4188-915f-fbed55a874a9 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1219 microversion: 2.1 time: 0.067766 Sep 17 04:34:01.703351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 305/616] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:01 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1219 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:01.831288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:01.831906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:01.838635 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:01.839120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:01.839566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:01.840257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:01.840708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:01.841122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:01.972696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:01.981652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:01.983090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:01.984253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8718c797-7bb7-4365-8f4c-d7c59f01b33d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.156987 Sep 17 04:34:01.984801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 312/617] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:01 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:02.432664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "name": "network-vif-plugged", "status": "completed", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:02.454748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:02.455233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:02.455688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:02.478377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] Creating event network-vif-plugged:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:34:02.484163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-6d8894d8-3a4f-429e-9fcd-c0e0d900e6ca req-3cda3700-5287-4f75-82bd-f2e67ad9abb1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056221 Sep 17 04:34:02.484768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 306/618] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:34:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:34:02.728270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:02.728898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:02.737109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:02.738144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:02.738768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:02.739573 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:02.740103 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:02.740600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:02.814358 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:02.823939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1d6f2349-cb1a-4951-8af4-51f6bcd3e607 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1271 microversion: 2.1 time: 0.101381 Sep 17 04:34:02.824511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 313/619] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:02 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1271 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:03.013053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:03.013650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:03.021384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:03.021869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:03.022447 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:03.023247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:03.023703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:03.024122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:03.039096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:03.168688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:03.178202 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:03.179952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:03.181276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bfa78a3e-e2a8-4191-8bd8-e263974cec8d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.174453 Sep 17 04:34:03.183653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 307/620] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:03 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:03.851668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:03.852659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:03.864001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:03.864804 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:03.865580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:03.866718 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:03.867333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:03.867759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:03.941925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:03.952922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a123a30a-814c-4ea7-ae56-de57e82e7121 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1271 microversion: 2.1 time: 0.107997 Sep 17 04:34:03.953656 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 314/621] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:03 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1271 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:04.214136 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:04.214920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:04.226397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:04.226882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:04.227317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:04.228008 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:04.228461 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:04.228885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:04.373822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:04.383775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:04.385242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:04.386382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4e4ad527-14be-43fc-92a2-6b17acac23f6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.179102 Sep 17 04:34:04.386938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 308/622] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:04 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:04.978827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:04.979370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:04.986066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:04.986558 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:04.986998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:04.987692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:04.988150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:04.988577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:05.050631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:05.058781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-72f53414-8b7e-4f12-b3b6-d24855329d27 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1289 microversion: 2.1 time: 0.084315 Sep 17 04:34:05.059339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 315/623] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:04 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1289 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:05.411879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:05.412633 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:05.419538 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:05.420104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:05.420668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:05.421471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:05.421996 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:05.422489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:05.631911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:05.641870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:05.643315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:05.644555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e7bd5cd1-bd46-4e0e-a12c-ad50b9aca88b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1804 microversion: 2.20 time: 0.236810 Sep 17 04:34:05.645483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 309/624] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:05 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1804 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:06.082411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:06.083056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:06.089806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:06.090297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:06.090741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:06.091468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:06.098927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:06.099389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:06.239887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:06.247988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c8b26222-538f-4f70-b422-934736484d3d tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.169683 Sep 17 04:34:06.248555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 316/625] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:06 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:06.670242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:06.671118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:06.678512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:06.679072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:06.679655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:06.680416 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:06.680930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:06.681419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:06.924687 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:06.935045 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:06.936484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:06.937709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ce73d61e-9733-4c78-af69-408d94322d52 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1791 microversion: 2.20 time: 0.271830 Sep 17 04:34:06.938240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 310/626] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:34:06 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1791 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:34:07.271408 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:07.272181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:07.279053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:07.279561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:07.279996 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:07.280688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:07.281162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:07.281611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:07.407607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56edc095-5585-49dd-9d48-b184f11c708a", "tag": "6743375d-5a28-4d3a-8f89-6396c2dfad6e"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:07.417397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:07.418052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:07.418591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:07.425815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:07.430572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] Creating event network-changed:6743375d-5a28-4d3a-8f89-6396c2dfad6e for instance 56edc095-5585-49dd-9d48-b184f11c708a on hv-cinder-85799 Sep 17 04:34:07.434885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4f931c81-d2ac-4fef-8dd0-ed61b89a09de tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.167368 Sep 17 04:34:07.435767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-71f92407-8728-4618-91fb-625e51f5d7ba req-0d3d69b3-4a1f-4329-8516-fb2cf4eeafa2 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100310 Sep 17 04:34:07.435920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 317/627] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:07 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:07.437792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 311/628] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:34:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:34:08.461511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:08.462140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:08.468915 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:08.469512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:08.469946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:08.470998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:08.471450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:08.471870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:08.614279 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:08.626265 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc368e73-bfc6-4a66-aabb-d13d3267803b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.170901 Sep 17 04:34:08.627120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 318/629] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:08 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:09.655883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:09.657052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:09.666549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:09.667010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:09.667459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:09.668137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:09.668583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:09.669007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:09.815250 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:09.826992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-109f93cb-5838-4c8b-aaa3-2bad57e553bc tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.179204 Sep 17 04:34:09.827734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 312/630] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:09 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:10.854340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:10.855006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:10.861689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:10.862174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:10.862596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:10.863286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:10.863738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:10.864165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:10.996882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:11.005285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dbaa06cf-29d3-4cc4-90ce-8e4a6f1af5ed tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.156761 Sep 17 04:34:11.005824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 319/631] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:10 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:12.032644 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:12.033662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:12.037714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:12.047102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:12.047909 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:12.048636 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:12.049844 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:12.050602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:12.051311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:12.186955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:12.198995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-88808f0e-039e-4680-b33e-2aa39da6b61b tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.172617 Sep 17 04:34:12.199857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 313/632] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:12 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:13.227593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:13.228201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:13.230122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:13.230764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:13.232304 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=51,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:13.238684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:13.239174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:13.239623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:13.240420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:13.240878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:13.241330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:13.372400 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:13.380691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c6b11bd6-ff7e-4fc2-8d5d-bce26ed9be25 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.157331 Sep 17 04:34:13.381264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 320/633] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:13 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:14.407882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:14.408836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:14.419898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:14.420712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:14.421494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:14.422634 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:14.423392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:14.424100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:14.559157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:14.566947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-10bdadf9-4a53-4bb2-8291-20ad9bed12ba tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.165652 Sep 17 04:34:14.567512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 314/634] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:14 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:14.942432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56edc095-5585-49dd-9d48-b184f11c708a", "name": "network-vif-plugged", "status": "completed", "tag": "6743375d-5a28-4d3a-8f89-6396c2dfad6e"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:14.952352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:14.952818 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:14.953276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:14.964060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] Creating event network-vif-plugged:6743375d-5a28-4d3a-8f89-6396c2dfad6e for instance 56edc095-5585-49dd-9d48-b184f11c708a on hv-cinder-85799 Sep 17 04:34:14.969112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-0269e82d-13fd-4e60-abb3-e0deec2b3b99 req-39af45d2-1279-4feb-ba6d-78eab9ea5cb6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091417 Sep 17 04:34:14.969722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 321/635] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:34:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:34:15.594016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:15.594939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:15.604912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:15.605428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:15.605863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:15.606551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:15.607004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:15.607431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:15.738291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:15.746468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-15bd4af7-0ea7-4072-aaf4-1454c94b2de8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.159069 Sep 17 04:34:15.747079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 315/636] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:15 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:16.779599 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:16.780576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:16.788403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:16.788959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:16.789545 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:16.790314 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:16.790765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:16.791182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:16.950547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:16.961767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4e4e28c1-3995-4053-a0ec-aeb208ccbb01 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.187386 Sep 17 04:34:16.962690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 322/637] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:16 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:17.996026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:17.996904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:18.007787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:18.008593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:18.009369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:18.010503 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:18.011257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:18.011962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:18.150181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:18.158112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ec65abe9-1885-4a63-ad71-1c836d2e35c8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1319 microversion: 2.1 time: 0.168989 Sep 17 04:34:18.158672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 316/638] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:17 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1319 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:19.184969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:19.185983 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:19.197110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:19.197620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:19.198055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:19.198743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:19.199194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:19.199618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:19.353339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:19.361260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-364fe28e-ab49-4dfa-8bc5-2ab3ab42dd33 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1509 microversion: 2.1 time: 0.182769 Sep 17 04:34:19.361823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 323/639] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:19 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1509 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:19.383753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c56faa93-4865-418e-bfec-e8b096a5ca82", "device": "/dev/sdb"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:19.386226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:19.393325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:19.393807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:19.394247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:19.394928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:19.395377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:19.395801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:19.695853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b5cc1071-a2d9-4e8b-8d9c-3584c3a4c115 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "POST /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.315628 Sep 17 04:34:19.696516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 317/640] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:34:19 2022] POST /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a/os-volume_attachments => generated 194 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:34:43.379192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:43.379743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:43.382048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=7 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:43.382690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=81 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:43.388458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:43.388934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:43.389392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:43.390074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:43.390516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:43.390933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:43.474968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lazy-loading 'services' on Instance uuid 56edc095-5585-49dd-9d48-b184f11c708a {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:34:43.555753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2506c97-e0d3-436d-9114-32e4bc5f5aa5 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "DELETE /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a/os-volume_attachments/c56faa93-4865-418e-bfec-e8b096a5ca82" status: 202 len: 0 microversion: 2.1 time: 0.180495 Sep 17 04:34:43.556508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 324/641] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:34:43 2022] DELETE /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a/os-volume_attachments/c56faa93-4865-418e-bfec-e8b096a5ca82 => generated 0 bytes in 182 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:34:45.746232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=31 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:45.747032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:34:45.752854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:45.753821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:45.764733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:45.765582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:45.766283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:45.767396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:45.768071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:45.768879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:45.794478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:34:45.876561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e7f31f9-fe08-4ed5-8572-fa47bd217e1f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "DELETE /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 204 len: 0 microversion: 2.1 time: 0.132838 Sep 17 04:34:45.877261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 318/642] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:34:45 2022] DELETE /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:34:45.887701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:45.888411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:45.894770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:45.895250 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:45.895682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:45.896475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:45.896965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:45.897467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:46.032523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:46.044323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f3c025e8-0c48-447f-b287-c7873e29b5f3 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1515 microversion: 2.1 time: 0.161502 Sep 17 04:34:46.045099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 325/643] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:45 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1515 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:34:47.070428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:47.071157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:47.077842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:47.078326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:47.078765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:47.079447 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:47.079904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:47.080325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:47.216667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:47.225441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-acf4db65-b799-4172-9fe4-5474574ca073 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1515 microversion: 2.1 time: 0.159400 Sep 17 04:34:47.226031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 319/644] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:47 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1515 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:48.249883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:48.250468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:48.257907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:48.258390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:48.258833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:48.259494 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:48.259977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:48.260410 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:48.402165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:48.410728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ce3fc02f-d578-48f3-9026-fb822635aff8 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1515 microversion: 2.1 time: 0.165302 Sep 17 04:34:48.411527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 326/645] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:48 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1515 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:48.650012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56edc095-5585-49dd-9d48-b184f11c708a", "name": "network-vif-deleted", "tag": "6743375d-5a28-4d3a-8f89-6396c2dfad6e"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:34:48.662092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:48.662878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:48.663297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:48.675886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] Creating event network-vif-deleted:6743375d-5a28-4d3a-8f89-6396c2dfad6e for instance 56edc095-5585-49dd-9d48-b184f11c708a on hv-cinder-85799 Sep 17 04:34:48.681610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-60983bb2-2e82-46e3-8175-3341fdc916d7 req-dd8a9b26-7165-4a9d-b1c6-29ab610a45a1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035252 Sep 17 04:34:48.682335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 320/646] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:34:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:34:49.434375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:49.435305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:49.442171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:49.442661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:49.443096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:49.443827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:49.444281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:49.444705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:49.477075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:49.477611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:49.478050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:49.478462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lazy-loading 'fault' on Instance uuid 56edc095-5585-49dd-9d48-b184f11c708a {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:34:49.524973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '166d99888375478e9cd8bf8dcb9d4190', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9d47a53afdbf404a829c119992fcb643', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:34:49.532155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-33e89907-2e92-416f-a6b2-e929ea43040f tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 200 len: 1309 microversion: 2.1 time: 0.102148 Sep 17 04:34:49.532726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 327/647] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:49 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 1309 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:34:50.560209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:34:50.561157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] [instance: 56edc095-5585-49dd-9d48-b184f11c708a] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:34:50.569856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:50.570520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:50.571106 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:50.572074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:34:50.572712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:34:50.573315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:34:50.589825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] HTTP exception thrown: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Request method failure captured: Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a HTTP/1.1 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU3CXnLmMd8_kli1tUzeSDDVRswtO5MCz6Xc0RVfUEVubONjZE2sBZmneO8bZkzgy1SIqmSE9bERdE1YbmpLalU1r1MeaanJei2Du8KMVRt2bELC2SPDp3akF9V05hImtp2emS1dHlLCMpJYuKu_KrbAYsl3pLQj8Nf5IR0xg6-QQGI Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 9d47a53afdbf404a829c119992fcb643 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:34:50.591442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_9d47a53afdbf404a829c119992fcb643"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/9d47a53afdbf404a829c119992fcb643"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 9d47a53afdbf404a829c119992fcb643 Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1455811953 Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesSnapshotTestJSON-1455811953-project Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 166d99888375478e9cd8bf8dcb9d4190 Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesSnapshotTestJSON-1455811953-project Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. Sep 17 04:34:50.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '56edc095-5585-49dd-9d48-b184f11c708a'} Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:34:50.594053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. Sep 17 04:34:50.595965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:34:50.597487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] Returning 404 to user: Instance 56edc095-5585-49dd-9d48-b184f11c708a could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:34:50.597487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8ff2dd66-1a15-4322-a25e-af5e6f25b860 tempest-VolumesSnapshotTestJSON-1455811953 tempest-VolumesSnapshotTestJSON-1455811953-project] 10.222.0.60 "GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a" status: 404 len: 111 microversion: 2.1 time: 0.039019 Sep 17 04:34:50.597487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 321/648] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:34:50 2022] GET /compute/v2.1/servers/56edc095-5585-49dd-9d48-b184f11c708a => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:03.491292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=10 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:03.492168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=29,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:03.492945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:35:03.493125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:35:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/ips (ip 10.222.0.60) !!! Sep 17 04:35:03.545623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:03.546115 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:03.552714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.553244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.553679 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.554348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.554793 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.555230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.579237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7b6c4f63-c96f-4c5a-aa63-a35181a93345 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/ips" status: 200 len: 148 microversion: 2.1 time: 0.103470 Sep 17 04:35:03.579793 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 328/649] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:35:03 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/ips => generated 148 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:03.597311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "02feb35d-2600-4285-bc1e-2644b9a0d936"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:03.600805 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:03.604261 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:03.605373 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:03.612389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.612900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.613362 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.614062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.614514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.614967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.695613 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: POST /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/os-volume_attachments HTTP/1.1 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 74 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:03.697281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {"volumeAttachment": {"volumeId": "02feb35d-2600-4285-bc1e-2644b9a0d936"}} Sep 17 04:35:03.698824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'server_id': 'acb9bb23-2e07-483b-b96a-e68d1cb95835', 'body': {'volumeAttachment': {'volumeId': '02feb35d-2600-4285-bc1e-2644b9a0d936'}}} Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:03.699719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:03.701043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 400 to user: Invalid volume: volume 02feb35d-2600-4285-bc1e-2644b9a0d936 is already attached to instances: 508f0590-1d24-42b5-9275-280c6cedff50 {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-34a9f723-003e-4c1e-8e1b-49670a338443 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.109214 Sep 17 04:35:03.702329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 322/650] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:35:03 2022] POST /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835/os-volume_attachments => generated 174 bytes in 111 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:35:03.708004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:03.708497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:03.714699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.715110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.715571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.716310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.716730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.717206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.744403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:35:03.827803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-80b6e53b-6bcb-4827-9c6f-f9fb28aec5de tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 204 len: 0 microversion: 2.1 time: 0.123384 Sep 17 04:35:03.828396 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 329/651] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:35:03 2022] DELETE /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:35:03.837043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:03.837934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:03.845240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.845768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.846203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.846912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:03.847366 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:03.847825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:03.982600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:03.994480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-032a0f67-2549-4a36-8084-146f8aef29eb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1678 microversion: 2.1 time: 0.161262 Sep 17 04:35:03.995248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 323/652] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:03 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:05.023569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:05.024349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:05.032831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:05.033353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:05.033819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:05.034537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:05.034989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:05.035415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:05.173451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:05.183012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c1a1c3ac-75fa-4094-bfbf-e37fbe527452 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1678 microversion: 2.1 time: 0.166399 Sep 17 04:35:05.183600 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 330/653] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:05 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1678 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:06.210951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:06.211934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:06.220660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:06.221122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:06.221575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:06.222257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:06.222706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:06.223125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:06.374189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:06.382539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-36802cb4-0e4f-4d6c-b40a-d445111bf02c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1678 microversion: 2.1 time: 0.178220 Sep 17 04:35:06.383105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 324/654] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:06 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1678 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:07.008106 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1481957129"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:07.008619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:07.014643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:07.015113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:07.015541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:07.016225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:07.016657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:07.017073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:07.147566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.network.security_group_api [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Removing security group 214549b0-5815-46e0-8206-a4e7a53a3e84 from port d54c1156-b70c-4805-bad8-7b71a108b5c0 Sep 17 04:35:07.406403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:07.406954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:07.413496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:07.413976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:07.414404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:07.415093 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:07.415534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:07.415953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:07.554679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:07.563177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e7e5a2c1-9ac1-4b2b-899f-273adbf013ec tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1678 microversion: 2.1 time: 0.160789 Sep 17 04:35:07.563737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 325/655] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:07 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1678 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:07.721803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fb0e6045-c674-4387-af5c-9c0c8ae56479 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action" status: 202 len: 0 microversion: 2.1 time: 0.824815 Sep 17 04:35:07.722421 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 331/656] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:35:06 2022] POST /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/action => generated 0 bytes in 826 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:07.786952 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:07.797514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:07.798027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:07.798435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:07.809919 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] Creating event network-changed:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:35:07.815141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-f8528011-a67d-4155-a0e0-2a1c8758c1d6 req-8292ee09-0c95-46b3-90eb-8e5e38282688 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088566 Sep 17 04:35:07.815924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 326/657] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:08.587378 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:08.588003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:08.594798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:08.595302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:08.595739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:08.596432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:08.596889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:08.597351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:08.740408 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:08.749311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-973e5f7e-342f-41f2-bf91-57c9b0d47458 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1678 microversion: 2.1 time: 0.166402 Sep 17 04:35:08.749876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 332/658] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:08 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1678 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:09.634751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:35:09.635361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:35:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.60) !!! Sep 17 04:35:09.641485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "acb9bb23-2e07-483b-b96a-e68d1cb95835", "name": "network-vif-deleted", "tag": "b715886e-d53b-4fcc-a366-f48a27e791df"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:09.681127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.682069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.682711 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.697278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] Creating event network-vif-deleted:b715886e-d53b-4fcc-a366-f48a27e791df for instance acb9bb23-2e07-483b-b96a-e68d1cb95835 on hv-cinder-85799 Sep 17 04:35:09.703185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-297ef64d-5537-4392-9014-e2ff33dc4203 req-101712ba-d94b-4b52-8f0c-f80e5059c611 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069677 Sep 17 04:35:09.704452 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 327/659] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:09.772549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:09.773963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:09.784043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.784949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.785414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.786127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.786568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.787167 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.850804 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.851548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.852344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.854289 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'fault' on Instance uuid acb9bb23-2e07-483b-b96a-e68d1cb95835 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:09.868195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:09.868987 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:09.877025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.878191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.879552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.880416 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:09.883271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:09.883990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:09.906031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:09.914068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-61ba1b62-41d8-4291-a031-2bb8c8764177 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 200 len: 1330 microversion: 2.1 time: 0.145818 Sep 17 04:35:09.927999 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 333/660] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:09 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 1330 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:10.051087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lazy-loading 'services' on Instance uuid b0c70917-7a03-48e3-be38-75f7f6add454 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:10.198176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1578c461-6b93-4804-8d4e-0cddf0c244c1 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments/0ba75f9b-6b57-4c6f-9290-3319432abf32" status: 202 len: 0 microversion: 2.1 time: 0.334041 Sep 17 04:35:10.201138 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 328/661] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:35:09 2022] DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments/0ba75f9b-6b57-4c6f-9290-3319432abf32 => generated 0 bytes in 336 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:10.207408 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:10.210040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:10.216606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:10.217391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:10.218073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:10.219355 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:10.220035 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:10.220498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:10.255086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-defd7adb-c254-454e-96a8-c6dc3aa5d6cf tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051759 Sep 17 04:35:10.256621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 334/662] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:10 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:10.941522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:10.942842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:10.949679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:10.950064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:10.950493 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:10.951201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:10.951652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:10.952078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:10.969893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 HTTP/1.1 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:10.972386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:35:10.974429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'acb9bb23-2e07-483b-b96a-e68d1cb95835'} Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:10.975515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:35:10.977172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:10.979208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:10.979208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-09adb2e0-b051-4175-a605-bf0a212ef343 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 404 len: 111 microversion: 2.1 time: 0.041855 Sep 17 04:35:10.980882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 329/663] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:10 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:10.988570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9dab6b43-debb-42ae-b462-8b7a9e5a2277 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '.version_select at 0x7fbc4f384af0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:11.022497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9dab6b43-debb-42ae-b462-8b7a9e5a2277 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-51696387" status: 202 len: 0 microversion: 2.1 time: 0.037966 Sep 17 04:35:11.023191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 335/664] 10.222.0.60 () {58 vars in 1215 bytes} [Sat Sep 17 04:35:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-51696387 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:11.275528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:11.278046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:11.285365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.286020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.286466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.287339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.287811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.288372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.323419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7a3cc1c3-3c74-436b-a261-9b623cb2c5bc tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059103 Sep 17 04:35:11.324423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 330/665] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:11 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:11.732144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:11.748557 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.749038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.749685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.761029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] Creating event network-changed:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:35:11.766570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-f652c847-f394-49aa-8223-5da6a8d113b5 req-e437970c-9818-434e-bde8-d80ca0a21f2d service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038147 Sep 17 04:35:11.767524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 336/666] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:11.781933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:11.782504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:11.788730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.789259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.789698 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.790392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.790853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.791275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.873544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'services' on Instance uuid 508f0590-1d24-42b5-9275-280c6cedff50 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:11.949739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-807ecbea-f86e-4ffb-a87f-0fabd7aa6e3e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments/02feb35d-2600-4285-bc1e-2644b9a0d936" status: 202 len: 0 microversion: 2.1 time: 0.171806 Sep 17 04:35:11.950768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 331/667] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:35:11 2022] DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments/02feb35d-2600-4285-bc1e-2644b9a0d936 => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:11.960319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:11.965853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:11.976956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.977556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.978297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:11.979254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:11.979714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:11.980137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.015749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-11bb0db0-428c-4ee8-9cfa-41ad59fb82cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059719 Sep 17 04:35:12.017495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 337/668] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:11 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:12.341231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:12.343840 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:12.351944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.352505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.353012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.353892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.354410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.354901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.385369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ed0dde7d-f01f-44b7-9944-def04edc4dd5 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051403 Sep 17 04:35:12.385904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 332/669] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:12 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:35:12.396634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:12.397127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:12.403554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.404061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.404496 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.405217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.405667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.406116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.433675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bbbe9e0a-0e35-4feb-a19d-5f4724115867"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:12.438197 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:12.447152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.447881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.448541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.449664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:12.450365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:12.451012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:12.487479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] HTTP exception thrown: Instance b0c70917-7a03-48e3-be38-75f7f6add454 is not attached to volume 0ba75f9b-6b57-4c6f-9290-3319432abf32 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Request method failure captured: Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments/0ba75f9b-6b57-4c6f-9290-3319432abf32 HTTP/1.1 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU02ExWkLFPXjqT0i8GqKXdM8Gl_9IjsCeZPHExrQExZB41cLOKMj-XbmH85rQ08w-3e8Dt7MNBH7VaIM4mb0UM1OQvAkygeXuWv9GcuxqKi668SuacFV7DYccjEpsprEaHEvPZNYlVo9FHUS2BoRuzL_rSpJoRjJVyrR0fAb0OGLwI Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 4ca16cc178624419954df6bad327f722 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:35:12.493143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/4ca16cc178624419954df6bad327f722"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/4ca16cc178624419954df6bad327f722"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_4ca16cc178624419954df6bad327f722"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/4ca16cc178624419954df6bad327f722"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 4ca16cc178624419954df6bad327f722 Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-TestMinimumBasicScenario-523132786-project Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: f6a5aff543234d7a9976c2e43453aef2 Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-TestMinimumBasicScenario-523132786-project Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:35:12.494736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance b0c70917-7a03-48e3-be38-75f7f6add454 is not attached to volume 0ba75f9b-6b57-4c6f-9290-3319432abf32 Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'server_id': 'b0c70917-7a03-48e3-be38-75f7f6add454', 'id': '0ba75f9b-6b57-4c6f-9290-3319432abf32'} Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 0ba75f9b-6b57-4c6f-9290-3319432abf32. Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:12.495596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0c70917-7a03-48e3-be38-75f7f6add454 is not attached to volume 0ba75f9b-6b57-4c6f-9290-3319432abf32 Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0c70917-7a03-48e3-be38-75f7f6add454 is not attached to volume 0ba75f9b-6b57-4c6f-9290-3319432abf32 Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:12.497100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Returning 404 to user: Instance b0c70917-7a03-48e3-be38-75f7f6add454 is not attached to volume 0ba75f9b-6b57-4c6f-9290-3319432abf32 {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:12.498214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-04aa88c2-03f2-4cb2-a4d6-40cbb065c809 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments/0ba75f9b-6b57-4c6f-9290-3319432abf32" status: 404 len: 154 microversion: 2.1 time: 0.102144 Sep 17 04:35:12.498214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 338/670] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:35:12 2022] DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454/os-volume_attachments/0ba75f9b-6b57-4c6f-9290-3319432abf32 => generated 154 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:12.844568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cb9d4899-79c1-4c83-8ac1-52a9833ce826 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.415218 Sep 17 04:35:12.845214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 333/671] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:35:12 2022] POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments => generated 194 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:13.030485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:13.032763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:13.041511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.041982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.042478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.043334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.043851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.044394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.074211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-956c1f00-7271-4614-a18c-0b1573e2d841 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048396 Sep 17 04:35:13.074772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 339/672] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:13 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:13.657844 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:13.659057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:13.666457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.666949 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.667380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.668081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.668540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.668967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.694207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:35:13.773821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-75f859c0-9ff1-4b68-bf50-a1254b899132 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 204 len: 0 microversion: 2.1 time: 0.119918 Sep 17 04:35:13.774500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 334/673] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:35:13 2022] DELETE /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:35:13.783193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:13.783841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:13.790487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.791133 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.791553 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.792254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:13.792740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:13.793168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:13.923808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:13.931917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c6672dd1-22c7-4301-a170-9e669f36ba90 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1554 microversion: 2.1 time: 0.152759 Sep 17 04:35:13.932501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 340/674] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:13 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1554 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:14.087807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:14.089955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:14.098164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.098658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.099113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.099805 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.108615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.108615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.130109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1e2e84b0-d7e4-4abe-853a-39a9f8abf651 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046737 Sep 17 04:35:14.130654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 335/675] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:35:14 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:35:14.196099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:14.196657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:14.203303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.203852 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.204391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.205167 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.205693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.206194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.233101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:35:14.310694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9fa7e427-e37b-4cf0-9982-ef2fe7bf3c65 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 204 len: 0 microversion: 2.1 time: 0.118807 Sep 17 04:35:14.311186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 341/676] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:35:14 2022] DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:35:14.319621 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:14.320257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:14.326421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.326896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.327329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.328011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.328471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.328891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.332888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=4,nova_cell1:UPDATE=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:14.466450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:14.475247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-11889615-f5e8-4d2e-975d-75da2edfe070 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1679 microversion: 2.1 time: 0.159515 Sep 17 04:35:14.475788 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 336/677] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:14 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1679 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:14.960192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:14.961266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:14.971886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.972684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.973504 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:14.974695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:14.975543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:14.976251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:15.122930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:15.131427 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-06adf567-ff22-498c-a945-7b05b5a6aafa tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1554 microversion: 2.1 time: 0.178523 Sep 17 04:35:15.132047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 342/678] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:14 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1554 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:15.499063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:15.499672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:15.506716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:15.507205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:15.507640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:15.508332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:15.508784 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:15.509304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:15.645056 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:15.653324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fac92361-e15d-4b39-9b06-9b1352352c93 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1679 microversion: 2.1 time: 0.158180 Sep 17 04:35:15.653888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 337/679] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:15 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1679 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:16.158234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:16.158942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:16.167237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:16.167830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:16.168363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:16.169254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:16.169817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:16.170336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:16.175374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:16.308690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:16.316966 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f7045101-f336-479a-84fa-01a74c6a6b14 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1554 microversion: 2.1 time: 0.164489 Sep 17 04:35:16.317590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 343/680] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:16 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1554 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:16.677459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:16.678076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:16.684893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:16.685417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:16.685851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:16.686537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:16.687007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:16.687508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:16.899569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:16.908243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2aea07d1-ed8d-40df-be3d-5e1ed1cad042 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1679 microversion: 2.1 time: 0.235034 Sep 17 04:35:16.909048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 338/681] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:16 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1679 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:17.348238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:17.348949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:17.356583 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:17.357206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:17.357624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:17.358309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:17.358750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:17.359165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:17.493284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:17.505147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-59e001f7-2eac-4207-b17c-712a18ad7608 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1554 microversion: 2.1 time: 0.161124 Sep 17 04:35:17.506143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 344/682] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:17 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1554 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:17.888383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0c70917-7a03-48e3-be38-75f7f6add454", "name": "network-vif-deleted", "tag": "d54c1156-b70c-4805-bad8-7b71a108b5c0"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:17.894096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:35:17.894283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:35:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.60) !!! Sep 17 04:35:17.903891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:17.904754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:17.905212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:17.918141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] Creating event network-vif-deleted:d54c1156-b70c-4805-bad8-7b71a108b5c0 for instance b0c70917-7a03-48e3-be38-75f7f6add454 on hv-cinder-85799 Sep 17 04:35:17.923012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-c0edd05a-367b-4387-bb16-86bd609c1f6a req-4e8ae700-44d1-4985-b74a-3b0967de4ac5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038425 Sep 17 04:35:17.923837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 339/683] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:17.932093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:17.932700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:17.938942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:17.939442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:17.940068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:17.940940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:17.941524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:17.941953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.205228 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:18.215796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2b332cf-eaa9-46b0-bc26-54b6e9ad9f30 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1679 microversion: 2.1 time: 0.287545 Sep 17 04:35:18.231247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 345/684] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:17 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1679 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:18.487901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:18.490178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:18.496287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.496735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.497256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.498000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.498458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.499064 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.522127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'flavor' on Instance uuid 260f958f-caf1-46d9-8a2e-74816fa1bfd6 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:18.527628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:18.528799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:18.535361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.535845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.536468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.537345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.537955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.538375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.573563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.574044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.574477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.574921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lazy-loading 'fault' on Instance uuid b0c70917-7a03-48e3-be38-75f7f6add454 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:18.633135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6a5aff543234d7a9976c2e43453aef2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ca16cc178624419954df6bad327f722', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:18.643460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e3b5015f-cf17-4282-9546-f0433af2ec0c tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 200 len: 1349 microversion: 2.1 time: 0.119402 Sep 17 04:35:18.656708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 346/685] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:18 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 1349 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:18.678474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b33ad04f-c120-45f6-96a2-3f1a314df10a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/action" status: 200 len: 32 microversion: 2.1 time: 0.194897 Sep 17 04:35:18.683730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 340/686] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:35:18 2022] POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/action => generated 32 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:35:18.692085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:18.692723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:18.706076 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.706576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.708565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.710009 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.710597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.712453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.720003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "508f0590-1d24-42b5-9275-280c6cedff50", "name": "network-vif-deleted", "tag": "976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:18.745272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:18.746574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:18.747037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:18.761430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] Creating event network-vif-deleted:976f2b6a-d9e7-4e56-ad03-1e8785f2c5ba for instance 508f0590-1d24-42b5-9275-280c6cedff50 on hv-cinder-85799 Sep 17 04:35:18.769860 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-10001a86-8e50-4e99-8b1a-376b99b29549 req-f49fd3cf-1e63-4252-8184-ba7511a9c11b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053439 Sep 17 04:35:18.770628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 341/687] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:19.001915 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:19.010932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-44b6da9c-ad23-4d93-a4a5-bced42ee57ad tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.325808 Sep 17 04:35:19.011689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 347/688] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:18 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:19.240584 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:19.241383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:19.248165 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:19.248579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:19.249013 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:19.249746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:19.250197 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:19.250621 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:19.282938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:19.283463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:19.284017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:19.284476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'fault' on Instance uuid 508f0590-1d24-42b5-9275-280c6cedff50 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:19.329656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:19.337756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-09c2bfbb-f31c-4da3-aee5-ac760921cbc6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 200 len: 1331 microversion: 2.1 time: 0.101385 Sep 17 04:35:19.338328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 342/689] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:19 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 1331 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:19.666905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:19.668131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] [instance: b0c70917-7a03-48e3-be38-75f7f6add454] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:19.679067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:19.679176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:19.679578 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:19.680363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:19.680810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:19.681611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:19.697404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] HTTP exception thrown: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Request method failure captured: Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 HTTP/1.1 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU02ExWkLFPXjqT0i8GqKXdM8Gl_9IjsCeZPHExrQExZB41cLOKMj-XbmH85rQ08w-3e8Dt7MNBH7VaIM4mb0UM1OQvAkygeXuWv9GcuxqKi668SuacFV7DYccjEpsprEaHEvPZNYlVo9FHUS2BoRuzL_rSpJoRjJVyrR0fAb0OGLwI Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 4ca16cc178624419954df6bad327f722 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:35:19.699079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/4ca16cc178624419954df6bad327f722"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/4ca16cc178624419954df6bad327f722"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_4ca16cc178624419954df6bad327f722"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/4ca16cc178624419954df6bad327f722"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 4ca16cc178624419954df6bad327f722 Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-TestMinimumBasicScenario-523132786 Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-TestMinimumBasicScenario-523132786-project Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: f6a5aff543234d7a9976c2e43453aef2 Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-TestMinimumBasicScenario-523132786-project Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. Sep 17 04:35:19.700572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': 'b0c70917-7a03-48e3-be38-75f7f6add454'} Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:19.701474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. Sep 17 04:35:19.702925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:35:19.705128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Returning 404 to user: Instance b0c70917-7a03-48e3-be38-75f7f6add454 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:19.705128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-edfb21e5-8577-4688-b587-3360200bad88 tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454" status: 404 len: 111 microversion: 2.1 time: 0.037485 Sep 17 04:35:19.705128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 348/690] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:19 2022] GET /compute/v2.1/servers/b0c70917-7a03-48e3-be38-75f7f6add454 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:19.709527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bd0daff6-b8bd-4d92-8133-6fc3374686eb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] Calling method '.version_select at 0x7fbc4f57bb80>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:19.737147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bd0daff6-b8bd-4d92-8133-6fc3374686eb tempest-TestMinimumBasicScenario-523132786 tempest-TestMinimumBasicScenario-523132786-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1388485820" status: 202 len: 0 microversion: 2.1 time: 0.031385 Sep 17 04:35:19.737891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 343/691] 10.222.0.60 () {58 vars in 1272 bytes} [Sat Sep 17 04:35:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1388485820 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:20.034491 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:20.035213 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:20.041567 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:20.042051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:20.042842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:20.044198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:20.044198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:20.044480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:20.178660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:20.189735 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2beba7a9-59ae-44c6-9253-18faf0c40a3d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.159080 Sep 17 04:35:20.190390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 349/692] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:20 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:20.362267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:20.362994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:20.369982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:20.370462 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:20.370887 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:20.371601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:20.372047 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:20.372555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:20.386005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 HTTP/1.1 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:20.387652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:35:20.389217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '508f0590-1d24-42b5-9275-280c6cedff50'} Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:20.390102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:35:20.391476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:20.392728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:20.392728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-baec1b8b-e30c-4ee8-8482-d5fd9b31d98a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 404 len: 111 microversion: 2.1 time: 0.031677 Sep 17 04:35:20.393203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 344/693] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:20 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:20.398048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b7c9f359-31e4-48a9-8547-1cfd63ff6ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '.version_select at 0x7fbc4f2f31f0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:20.417738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b7c9f359-31e4-48a9-8547-1cfd63ff6ba1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-272240666" status: 202 len: 0 microversion: 2.1 time: 0.023451 Sep 17 04:35:20.418313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 350/694] 10.222.0.60 () {58 vars in 1218 bytes} [Sat Sep 17 04:35:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-272240666 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 17 04:35:21.149176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b7be2424-2957-4055-b777-dc81cd87f0d1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: .version_select at 0x7fbc4f567ee0>, body: {"keypair": {"name": "tempest-keypair-1441753317", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG1FPOND8gcm9R7/pAoHlIv6B5yj5qEUrPjDy4sZ400/6vDlyVQoOJcb1qiZZstYjl/GN1XCSmrc9vLHn8av9Qhbqpf0C5bSgFoEkzjjQmD4cFPxhMEx3JhQWAz0uE8JqA=="}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:21.164924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-b7be2424-2957-4055-b777-dc81cd87f0d1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:35:21.168843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-b7be2424-2957-4055-b777-dc81cd87f0d1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:35:21.191251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b7be2424-2957-4055-b777-dc81cd87f0d1 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.045922 Sep 17 04:35:21.191831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 345/695] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:35:21 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:21.215297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:21.216011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:21.223265 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:21.223868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:21.224274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:21.225010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:21.225539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:21.225928 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:21.429778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:21.439898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4e850ac1-8382-4bb2-835b-a7459c96a2a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.229768 Sep 17 04:35:21.440548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 351/696] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:21 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:22.464611 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:22.465246 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:22.480541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:22.481132 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:22.481709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:22.482834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:22.483315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:22.483905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:22.747128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:22.757167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-81128f8e-e55f-4359-bede-d1e4d4714893 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.296999 Sep 17 04:35:22.758180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 346/697] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:22 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:23.402633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-232770466", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5d741fbf-1171-42a3-8de2-aef8bbcc1581"}], "security_groups": [{"name": "tempest-securitygroup--12919276"}], "key_name": "tempest-keypair-1441753317", "user_data": "}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:23.539536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] validate_networks() for [('5d741fbf-1171-42a3-8de2-aef8bbcc1581', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:35:23.781587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:23.782746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:23.791648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:23.792982 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:23.794593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:23.795782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:23.795782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:23.796428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.008045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:35:24.008459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:35:24.009068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:35:24.009502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:35:24.009908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:35:24.019403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.019893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.020326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.021101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.021587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.022036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.041419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:35:24.050625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:35:24.052324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:24.063769 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7c943d69-5aab-43ff-8a2f-6c11c3447258 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.287017 Sep 17 04:35:24.064971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 347/698] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:23 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:24.087149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:35:24.090201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:35:24.150093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-918a7ae9-2cfc-46c0-b9a7-5c787ad2373f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.751824 Sep 17 04:35:24.151362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 352/699] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:35:23 2022] POST /compute/v2.1/servers => generated 400 bytes in 754 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:35:24.164693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e20ff5a1-3a24-46ee-b77a-e370a12046d6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:24.165689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e20ff5a1-3a24-46ee-b77a-e370a12046d6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:24.265612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e20ff5a1-3a24-46ee-b77a-e370a12046d6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:24.290244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e20ff5a1-3a24-46ee-b77a-e370a12046d6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1241 microversion: 2.1 time: 0.128234 Sep 17 04:35:24.291396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 348/700] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:24 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1241 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:24.553436 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:24.554000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:24.562124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.562612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.563042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.563728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.564341 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.564759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.591896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:35:24.684074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3239da8f-2eba-4e9b-a51a-3e5b73e89ceb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 204 len: 0 microversion: 2.1 time: 0.134827 Sep 17 04:35:24.685182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 353/701] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:35:24 2022] DELETE /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:35:24.693854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:24.694521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:24.702029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.702727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.703380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.704095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:24.704530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:24.704947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:24.843271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:24.851399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-244eddbe-15d7-4379-898f-53df39ffa4f5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1578 microversion: 2.1 time: 0.162044 Sep 17 04:35:24.851970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 349/702] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:24 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1578 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:25.092014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:25.092815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:25.099694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.100262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.100775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:25.101590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.102135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.102615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:25.296191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:25.304795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-442bb2da-1f4d-4fd3-bbd8-14b1b1f92bea tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.218914 Sep 17 04:35:25.305429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 354/703] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:25 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:25.314268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:25.315017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:25.321430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.322028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.322485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:25.323174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.323628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.324053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:25.401011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:25.410637 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4b9fbb8e-0227-4933-b782-a1b8c96a4425 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1237 microversion: 2.1 time: 0.100490 Sep 17 04:35:25.412083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 350/704] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:25 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1237 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:25.875221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:25.876049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:25.882806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.883388 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.883902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:25.884670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:25.885223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:25.885723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:26.024807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:26.035535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d17586b8-43aa-4134-986c-5874c5fd37ed tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1578 microversion: 2.1 time: 0.163574 Sep 17 04:35:26.036777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 355/705] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:25 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1578 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:26.335689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:26.336315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:26.353707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:26.354480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:26.355091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:26.355995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:26.356708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:26.357329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:26.435165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:26.435785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:26.439171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=3,nova_api:DELETE=2,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:26.444107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:26.448397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:26.448844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:26.449623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:26.450101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:26.450506 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:26.521754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:26.530517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b21c4392-25b0-4bef-9edc-c302a5b3e545 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1293 microversion: 2.1 time: 0.099582 Sep 17 04:35:26.531097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 356/706] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:26 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1293 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:26.625976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:26.636562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-507de9de-404b-4d05-a3b6-703f3ea8c19f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.306967 Sep 17 04:35:26.637709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 351/707] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:26 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:27.060080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:27.062414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:27.074107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.074848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.079001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.080142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.080794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.081636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.335925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:27.344699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-21f76275-a6a7-417a-a622-3ebc745f9c47 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1578 microversion: 2.1 time: 0.288687 Sep 17 04:35:27.357751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 357/708] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:27 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1578 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:27.554176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:27.556386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:27.564494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.565320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.565983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.566882 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.567534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.568191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.662064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:27.663080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:27.674187 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.676398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.676909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.677704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:27.678359 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:27.679010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:27.699393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:27.711968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c0c2862a-daa4-4021-a3a8-d11447bb4f3d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1299 microversion: 2.1 time: 0.161864 Sep 17 04:35:27.712569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 352/709] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:27 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1299 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:28.148292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f28c067-5b02-4cb7-8284-c59d2f694528", "name": "network-vif-deleted", "tag": "fa6be46f-8a19-4a8d-84f9-67c67c4d32be"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:28.158850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:28.159609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:28.160235 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:28.179348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] Creating event network-vif-deleted:fa6be46f-8a19-4a8d-84f9-67c67c4d32be for instance 8f28c067-5b02-4cb7-8284-c59d2f694528 on hv-cinder-85799 Sep 17 04:35:28.180833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:28.192028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d6b478eb-a6a8-4021-abe6-1cb489dba5b0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.535723 Sep 17 04:35:28.192782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 358/710] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:27 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:28.193432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-3e87cc37-4dd7-4bfd-ad51-cb0776ffb138 req-67d1dc8d-8297-4d45-990f-40508138f91d service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048927 Sep 17 04:35:28.194124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 353/711] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:28.368782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:28.369424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:28.375730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:28.376216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:28.376652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:28.377379 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:28.377835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:28.378256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:28.484001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:28.493594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-307d225a-743c-4523-afd4-dc3e058c3fda tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 200 len: 1225 microversion: 2.1 time: 0.128641 Sep 17 04:35:28.496440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 359/712] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:28 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 1225 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:28.741619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:28.744195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:28.752495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:28.752976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:28.753548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:28.754625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:28.755235 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:28.755803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:28.982404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:28.995232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5f6980e7-17cf-427b-bf43-c444dc9a7bf6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.263209 Sep 17 04:35:28.995970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 354/713] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:28 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:29.216878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:29.218232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:29.225451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:29.226191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:29.226624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:29.227317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:29.227801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:29.228223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:29.233535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:29.430708 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:29.445224 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d7140496-fb63-4f51-aefa-4e5fa622a8bc tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.227944 Sep 17 04:35:29.446431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 360/714] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:29 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:29.520015 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:29.520729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:29.528065 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:29.528893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:29.529515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:29.530266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:29.530719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:29.531177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:29.545163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] HTTP exception thrown: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Request method failure captured: Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 HTTP/1.1 Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU10aVklLgmHscuVU-P_qRXWpxyIi6BuYozKUiz_g1RNdopf3m3ZxkfLshUkyOZN1IrJrzlPmza_SVVEeFGpA58DRH_yMJIW0H3GhBIg4U_sVvt42SOSveNU04ThmFoyxguY_L6uEo_JnxLDGBqDu_NVF8ThtqeHvY2olc3IJ48zI4o Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:29.547048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ff241cbd3f7c47328ac5ef2ed4c66e18 Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:35:29.548408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '8f28c067-5b02-4cb7-8284-c59d2f694528'} Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:29.549326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:35:29.550559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:29.551661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Returning 404 to user: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:29.551661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3a8588f7-bd12-4544-ad9c-4fbdbf91e547 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 404 len: 111 microversion: 2.1 time: 0.032353 Sep 17 04:35:29.563966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 355/715] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:29 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:35:29.672035 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-379058153", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-623666681", "security_groups": [{"name": "tempest-secgroup-smoke-2116056742"}], "block_device_mapping_v2": [{"uuid": "60182b5d-bc1a-403c-8972-625aa7fb815e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e322731d-9c63-4618-8051-e8dea6bb2db6"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:29.804461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] validate_networks() for [('e322731d-9c63-4618-8051-e8dea6bb2db6', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:35:29.955727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8bc28c77-6b66-4f4e-819e-c6047e0e6ddd", "tag": "fa109493-7edb-4c37-bc35-5135e4fef636"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:29.967964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:29.968475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:29.969432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:29.984740 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] Creating event network-changed:fa109493-7edb-4c37-bc35-5135e4fef636 for instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd on hv-cinder-85799 Sep 17 04:35:29.989762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-fb595a93-2ef9-418a-9763-4bfa5f9c2603 req-cf903d95-cd11-4878-8cff-16cede39c9bf service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037673 Sep 17 04:35:29.990709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 356/716] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:30.020101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:30.020826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:30.027145 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.027629 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.028062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.028752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.029235 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.029658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.049271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:30.203203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:35:30.203657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:35:30.204128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:35:30.204554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:35:30.204937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:35:30.214946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.215522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.216381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.217242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.217834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.218254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.242637 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:35:30.245570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:30.247718 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for user ff241cbd3f7c47328ac5ef2ed4c66e18 and project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:35:30.253653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5bb9d87f-cd6c-45c8-8d22-cb38a7ed2e20 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.237771 Sep 17 04:35:30.254436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 357/717] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:30 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:30.307132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:35:30.309415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='60182b5d-bc1a-403c-8972-625aa7fb815e',volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:35:30.442985 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aeebdd36-95ab-4bc9-b4ad-0ed38ee060f1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.776164 Sep 17 04:35:30.450023 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 361/718] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:35:29 2022] POST /compute/v2.1/servers => generated 406 bytes in 778 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:35:30.456383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b0df22f2-8eab-4d97-aaf0-8debe1c3463f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:30.457141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b0df22f2-8eab-4d97-aaf0-8debe1c3463f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:30.470340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:30.471334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:30.478903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.479415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.479870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.480537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:30.481319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:30.481898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:30.532767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b0df22f2-8eab-4d97-aaf0-8debe1c3463f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:30.542464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b0df22f2-8eab-4d97-aaf0-8debe1c3463f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1093 microversion: 2.1 time: 0.089721 Sep 17 04:35:30.543896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 358/719] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:30 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1093 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:30.629731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:30.638184 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-986ee56c-8d85-440d-bcab-7ffed3603ae6 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.172455 Sep 17 04:35:30.638754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 362/720] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:30 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:31.276775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:31.277454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:31.281004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:31.285858 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.286357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.286771 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.287466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.287917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.288338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.429474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:31.440369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e972567e-51a3-4947-b8a7-e3bd72427863 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.167588 Sep 17 04:35:31.441099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 359/721] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:31 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:31.570037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:31.570932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:31.579377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.579864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.580301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.580991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.581475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.581906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.652431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:31.660736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e0987c6e-a823-4101-b48f-adca19a9577c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1143 microversion: 2.1 time: 0.097172 Sep 17 04:35:31.661524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 363/722] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:31 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1143 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:31.662578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:31.663303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:31.669715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.670324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.671043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.671536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:31.672033 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:31.672459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:31.802825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:31.811696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e5479aea-8bb0-4c4e-aedd-7f9975796600 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.153339 Sep 17 04:35:31.812265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 360/723] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:31 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:32.468975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:32.469621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:32.476242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.476739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.477205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.477918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.478403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.478812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.613031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:32.621362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d7909436-54c9-441b-ace4-2ab3aac0a64d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.157724 Sep 17 04:35:32.622029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 364/724] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:32 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:32.686661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:32.687276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:32.694130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.694618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.695053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.695743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.696200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.696622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.771701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:32.783339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1b1fdb2d-25f2-4e7e-893e-2e053a7a9db2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1143 microversion: 2.1 time: 0.101964 Sep 17 04:35:32.784117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 361/725] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:32 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1143 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:32.835326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:32.836232 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:32.843414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.844246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.844674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.845394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:32.845842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:32.846291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:32.998518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:33.009531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9fac5374-283c-4bee-9d75-9ca237d6fd2b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.177948 Sep 17 04:35:33.010844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 365/726] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:32 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:33.644774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:33.645434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:33.651649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:33.652139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:33.652575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:33.653303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:33.653761 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:33.654189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:33.810990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:33.811850 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:33.819851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:33.820672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:33.821632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:33.822449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:33.823011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:33.823587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:33.869547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:33.877294 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7af89397-8ccc-4d62-b784-4e5dd4f828c0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.236498 Sep 17 04:35:33.878121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 362/727] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:33 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:33.896851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:33.904992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e74ce836-0303-4c75-b6a0-aac13a69e528 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1209 microversion: 2.1 time: 0.099636 Sep 17 04:35:33.905610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 366/728] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:33 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1209 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:34.034074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:34.034683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:34.040877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.041401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.041844 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:34.042543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.043001 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.043427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:34.249012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:34.257790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f967afbf-a855-41d6-b7f1-f83951f30b58 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.227830 Sep 17 04:35:34.258347 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 363/729] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:34 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:34.900530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:34.901283 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:34.908398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.909183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.909616 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:34.910296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.910740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.911155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:34.930408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:34.931105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:34.937911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.938601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.939237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:34.939938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:34.940664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:34.941282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:35.052508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:35.062648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9c48c418-469c-44d3-94f7-ac8e1eea8ca6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.166006 Sep 17 04:35:35.063539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 367/730] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:34 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:35.202808 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:35.211373 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-90a15aad-4950-4c2a-8e5d-235804efcf17 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.284764 Sep 17 04:35:35.212306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 364/731] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:34 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:35.283104 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:35.283736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:35.290694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:35.291138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:35.291784 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:35.292775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:35.293207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:35.293527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:35.394655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:35.408454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:35.410040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:35.410825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:35.426723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] Creating event network-changed:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:35:35.432466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-b1156de7-6cf7-46fa-a52f-634dd5c379a3 req-dc67ad61-fc27-4336-a42c-f260434ea3b7 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042647 Sep 17 04:35:35.433961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 365/732] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:35:35.454733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:35.463128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f0468110-b7a5-4b9a-b8c0-2948d376bae5 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.184768 Sep 17 04:35:35.464245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 368/733] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:35 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:36.087890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:36.088650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:36.095672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.096163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.096578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.097392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.097849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.098266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.236459 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:36.237117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:36.238375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:36.243962 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.244486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.244924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.245722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.246177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.246293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c86390f7-8791-4b31-9e39-62637fff06e0 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.163319 Sep 17 04:35:36.246604 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.246923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 366/734] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:36 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:36.385627 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:36.394247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-86c93dc5-0704-4dd6-9bad-a8b720794492 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.161970 Sep 17 04:35:36.394799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 369/735] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:36 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:36.489281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:36.489951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:36.498140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.498632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.499074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.499770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.500224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.500656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.651171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:36.660777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-81cd05b8-8a81-41e1-b51c-f3bae0584510 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.176894 Sep 17 04:35:36.661975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 367/736] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:36 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:36.964862 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:36.965845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:36.972581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.973064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.973517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:36.974200 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:36.974641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:36.975059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.004338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5469695f-4938-4646-a0c7-8dc54df0c673 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments/0d8ba18e-24b7-457e-8524-26fd4cb189bf" status: 200 len: 194 microversion: 2.20 time: 0.046318 Sep 17 04:35:37.004898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 370/737] 10.222.0.60 () {60 vars in 1454 bytes} [Sat Sep 17 04:35:36 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments/0d8ba18e-24b7-457e-8524-26fd4cb189bf => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:35:37.078152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:37.079081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:37.085827 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.086320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.086753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.087458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.087910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.088333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.173349 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lazy-loading 'services' on Instance uuid 0043b5dd-c019-4a76-b2d0-4c68982cac21 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:37.250694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8bc28c77-6b66-4f4e-819e-c6047e0e6ddd", "name": "network-vif-plugged", "status": "completed", "tag": "fa109493-7edb-4c37-bc35-5135e4fef636"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:37.251289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-76354681-6b03-4de7-9de6-acf8d681a2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments/0d8ba18e-24b7-457e-8524-26fd4cb189bf" status: 202 len: 0 microversion: 2.20 time: 0.176179 Sep 17 04:35:37.253149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 368/738] 10.222.0.60 () {60 vars in 1457 bytes} [Sat Sep 17 04:35:37 2022] DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments/0d8ba18e-24b7-457e-8524-26fd4cb189bf => generated 0 bytes in 180 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:35:37.262552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:37.266658 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:37.267830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.268612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.269044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.274458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.275190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.276843 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.277792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.278541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.279158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.287319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] Creating event network-vif-plugged:fa109493-7edb-4c37-bc35-5135e4fef636 for instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd on hv-cinder-85799 Sep 17 04:35:37.292786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-cda07e04-aebc-45bd-937b-1753c49f0d8a req-cf6f2201-0c56-434e-912b-843ea743494c service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047252 Sep 17 04:35:37.294455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 371/739] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:37.305453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:37.306494 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:37.314895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.315395 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.315826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.317146 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.317726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.318153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.335278 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e513ff25-f79b-449a-8dc7-68eb983c7e2f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.077349 Sep 17 04:35:37.335863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 369/740] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:35:37 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:35:37.417402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:37.418050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:37.424287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.424776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.425247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.425945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.426402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.426826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.458582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:37.466409 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a6b6de90-b05d-488f-ba21-fcb2948d4f9b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.170442 Sep 17 04:35:37.466986 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 372/741] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:37 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:37.562303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:37.570142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bb6e9ab0-711b-46d3-8282-2313b3e622ef tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.156706 Sep 17 04:35:37.570737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 370/742] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:37 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:37.686853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:37.687453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:37.693888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.694370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.694794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.695473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:37.695920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:37.696337 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:37.838984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:37.847936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8f8a7d62-efee-4309-b1b6-62254634a67e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1735 microversion: 2.1 time: 0.166513 Sep 17 04:35:37.848496 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 373/743] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:37 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:38.348957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:38.351179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:38.358071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.358559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.358994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.359774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.360211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.360652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.391702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-51dabbb9-b604-4cb3-92c3-b06f65a8db44 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046826 Sep 17 04:35:38.392292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 371/744] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:35:38 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:35:38.492469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:38.493535 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:38.509151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.509960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.511008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.512150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.512904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.513658 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.593048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:38.594038 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:38.601000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.601521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.601955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.602649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.603100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.603524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.647105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:38.653033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-eba03ef5-bbbb-4093-a601-ff10b899535f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.166974 Sep 17 04:35:38.653616 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 374/745] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:38 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:38.739512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:38.747910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cbd7737d-d149-493c-81d2-9aac926035e0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.158737 Sep 17 04:35:38.748561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 372/746] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:38 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:38.878701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:38.879307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:38.885468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.885945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.886398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:38.887073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:38.887517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:38.887938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.019165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:39.028045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-01b10766-d65d-41d4-97d4-98578a78772d tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1715 microversion: 2.1 time: 0.153323 Sep 17 04:35:39.028613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 375/747] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:38 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1715 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:39.142016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:39.142508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.149035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.149552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.149990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.150695 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.151170 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.151591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.237650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] HTTP exception thrown: Cannot 'detach_volume' instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 while it is in vm_state rescued Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Request method failure captured: Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments/bbbe9e0a-0e35-4feb-a19d-5f4724115867 HTTP/1.1 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU1VyO3YwjgPsO3zAE4FZmsyWEcgb5B0Oy5RcSv7jAJnr5DJfg3c80tKcUCh5_S8hLjEcJ5qglAE8bqRQMN3PZ86lwmiGwGAxYPEQcqx_A0AYCoaReV-dg5nm3Mg-oVytdBrAl3CriJDBmv83kbBOhg2wmCsC9XwgpE4vLYHXFOTWwg Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:39.239308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 5bb515269d1a45f594f3d297c7fd267e Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:39.240963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Cannot 'detach_volume' instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 while it is in vm_state rescued Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'server_id': '260f958f-caf1-46d9-8a2e-74816fa1bfd6', 'id': 'bbbe9e0a-0e35-4feb-a19d-5f4724115867'} Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:39.241891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 while it is in vm_state rescued Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 while it is in vm_state rescued Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:39.243424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Returning 409 to user: Cannot 'detach_volume' instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 while it is in vm_state rescued {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:39.245009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fc2433cc-8210-4454-a485-b359cb7d6049 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments/bbbe9e0a-0e35-4feb-a19d-5f4724115867" status: 409 len: 152 microversion: 2.1 time: 0.102853 Sep 17 04:35:39.245009 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 373/748] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:35:39 2022] DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments/bbbe9e0a-0e35-4feb-a19d-5f4724115867 => generated 152 bytes in 104 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 17 04:35:39.249868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:39.250390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.256775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.257303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.257737 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.258417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.258861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.259270 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.330833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-83bbe309-817d-4106-bc50-d562fa433cdf tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/action" status: 202 len: 0 microversion: 2.1 time: 0.084722 Sep 17 04:35:39.331494 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 376/749] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:35:39 2022] POST /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/action => generated 0 bytes in 86 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:35:39.339709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:39.340381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.346883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.347365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.347801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.348490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.348943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.349388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.404896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:39.407089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.413399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.413900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.414326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.415016 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.415465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.415885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.419732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:39.489990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:39.499073 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0a1326ec-15e4-4260-acb8-73e8f7d600ba tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.163127 Sep 17 04:35:39.499914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 374/750] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:39 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:39.606159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8da00b55-32a5-43ce-827b-073ac54c158d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.205753 Sep 17 04:35:39.606770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 377/751] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:35:39 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 197 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 17 04:35:39.675344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:39.675976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.682338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.682818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.683248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.683925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.684370 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.684787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.688880 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:39.771305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:39.771912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:39.778430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.779028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.779529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.780282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:39.780793 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:39.781389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:39.896178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:39.903913 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4a9c695d-a551-4b1e-ad3e-6c6eaa4090dc tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1365 microversion: 2.1 time: 0.232496 Sep 17 04:35:39.904473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 375/752] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:39 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1365 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:40.005626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:40.026395 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a03aa099-9e9c-4971-8dde-c4d666afcd87 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.258691 Sep 17 04:35:40.033174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 378/753] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:39 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:40.527132 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:40.528142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:40.536322 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.536806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.537275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:40.537956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.538402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.538824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:40.539254 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=8 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:40.618933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:40.621386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:40.624563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=4 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:40.625312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=12 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:40.630221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.630787 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.631292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:40.632070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.632612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.633122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:40.670303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3c9a7317-f400-4561-98af-74ffb799ccd2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055293 Sep 17 04:35:40.670914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 379/754] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:35:40 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:35:40.674471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:40.683014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7c328dd1-ad2d-4bb5-8129-78d28a31e479 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.162461 Sep 17 04:35:40.683577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 376/755] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:40 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:40.930044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:40.930920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:40.939642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.940251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.940871 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:40.941914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:40.942561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:40.943104 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.085588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:41.093983 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-39de72a4-d310-4c22-b4ed-77eee2651bba tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1554 microversion: 2.1 time: 0.169903 Sep 17 04:35:41.094566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 380/756] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:40 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1554 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:41.113547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.114160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.116088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.116608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.120288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.120777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.121274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.121964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.122410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.122561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.122838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.123050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.123481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.124173 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.124613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.125033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.206064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bbd668fe-5702-44cc-8933-13e62f5338a4 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093602 Sep 17 04:35:41.206995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 381/757] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:35:41 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:41.254047 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:41.261857 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1fd42f66-0621-4693-b79b-eae8edce3ec8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.152255 Sep 17 04:35:41.262469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 377/758] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:41 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:41.684089 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.686467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.692685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.693337 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.693877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.694674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.695226 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.695736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.707152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.710727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.721636 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.722385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.723123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.723980 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.725245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.726375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.731288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-873d69e1-d9a2-426c-9963-8a3ff9ead420 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.050766 Sep 17 04:35:41.733463 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 382/759] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:35:41 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:41.810343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.811155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.817927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.818518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.819063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.819849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.820790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.821021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.847743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:35:41.938584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-44ea14a6-97b4-45a0-b504-101bd408c35b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 204 len: 0 microversion: 2.20 time: 0.131485 Sep 17 04:35:41.943372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 383/760] 10.222.0.60 () {60 vars in 1280 bytes} [Sat Sep 17 04:35:41 2022] DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 17 04:35:41.949847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:41.950604 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:41.956645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:41.958050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.958577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.958979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.959675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:41.960477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:41.960925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:41.965723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d1b1c583-fcaf-4597-9ecc-a82440beedc8 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.263178 Sep 17 04:35:41.966522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 378/761] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:41 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:42.195168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:42.204042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:42.205548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:42.206721 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-70693d6f-2805-49c9-9a28-1a2006167818 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.261666 Sep 17 04:35:42.207516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 384/762] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:41 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:42.284862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:42.285476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:42.291540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:42.292022 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:42.292459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:42.293143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:42.293675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:42.294052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:42.429437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:42.438622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-025c7600-bb7e-4cfb-8c22-8435a517d0b8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.156925 Sep 17 04:35:42.438622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 379/763] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:42 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:42.461437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8bc28c77-6b66-4f4e-819e-c6047e0e6ddd", "tag": "fa109493-7edb-4c37-bc35-5135e4fef636"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:42.472222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:42.472696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:42.473144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:42.486696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] Creating event network-changed:fa109493-7edb-4c37-bc35-5135e4fef636 for instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd on hv-cinder-85799 Sep 17 04:35:42.493141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-5a152d9d-5b4a-4b00-9316-280d14ca8cd0 req-4ca1cb53-6282-47a1-a997-1abc1cf15ab6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035266 Sep 17 04:35:42.493903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 385/764] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:42.990832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:42.991441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:42.998467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:42.998958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:42.999392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.000087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:43.000549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:43.000975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.132670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:43.141167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d2b76be9-d803-483d-b7e9-3f348c587084 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.154154 Sep 17 04:35:43.141969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 380/765] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:42 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:43.232734 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:43.233489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:43.241504 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:43.242234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:43.242907 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.243950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:43.244657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:43.245325 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.433313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:43.442993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:43.444560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:43.445842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-09dcee5e-1a0e-4fda-a062-78914d0b2474 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.217487 Sep 17 04:35:43.446403 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 386/766] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:43 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:43.462502 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:43.463042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:43.469504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:43.469988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:43.470414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.471207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:43.471846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:43.472444 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:43.683393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:43.691900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5ab75c38-ddfc-483d-8305-187d1cff6ed3 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.233268 Sep 17 04:35:43.692585 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 381/767] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:43 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:44.168878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:44.169798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:44.180410 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.181186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.181993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.183134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.183892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.184642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.324765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:44.333452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4839ee58-40c2-4455-b2de-c94112f8aae9 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.171281 Sep 17 04:35:44.334019 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 387/768] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:44 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:44.472548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:44.473256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:44.480218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.480700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.481127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.481844 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.482297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.482716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.715858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:44.716784 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:44.723813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.724305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.724742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.725492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:44.725949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:44.726372 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:44.864053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:44.871894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a2576434-23f7-4ba8-bdfa-25cb75d70069 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.160086 Sep 17 04:35:44.872472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 388/769] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:44 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:44.874260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:44.882943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:44.884479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:44.885878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8ecc9431-c79c-4eb1-b108-017e856eaa6c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.417711 Sep 17 04:35:44.886520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 382/770] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:44 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:45.361622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:45.362631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:45.374057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.374866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.375596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:45.376802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.377623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.378342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:45.561671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:45.570174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bfb3e328-69de-4408-b20b-aeb01161b8d2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1723 microversion: 2.1 time: 0.215232 Sep 17 04:35:45.570711 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 389/771] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:45 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1723 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:45.896225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:45.896825 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:45.903479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.904640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.904640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:45.910012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:45.910656 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:45.913785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.914353 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.914863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:45.916514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.917027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.917485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:45.918170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:45.918733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:45.919819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.053020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:46.062029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:46.063507 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:46.064690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a027a0d7-f345-4a13-9391-92656f949799 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1719 microversion: 2.20 time: 0.158603 Sep 17 04:35:46.065277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 390/772] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:45 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1719 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:46.148768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:46.157102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-084f67e8-86ed-4bd3-97a8-8ad38ec2f689 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.264823 Sep 17 04:35:46.157648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 383/773] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:45 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:46.595029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:46.596444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:46.603694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:46.604074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:46.604516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.605310 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:46.605935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:46.606521 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.667946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0043b5dd-c019-4a76-b2d0-4c68982cac21", "name": "network-vif-deleted", "tag": "987d1030-ee5a-46f3-b114-6a60faeba2a9"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:46.678366 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:46.678969 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:46.679463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.691579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] Creating event network-vif-deleted:987d1030-ee5a-46f3-b114-6a60faeba2a9 for instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 on hv-cinder-85799 Sep 17 04:35:46.703243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-6ae2e73f-5bab-4e98-90f3-bc18e1c04b23 req-a44cf375-544f-462d-acdf-0b1c15149c78 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038965 Sep 17 04:35:46.704433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 384/774] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:46.835997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:46.845660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6679d99a-30ff-4050-b2e2-cc972846062b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1729 microversion: 2.1 time: 0.255102 Sep 17 04:35:46.847450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 391/775] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:46 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1729 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:46.870364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:46.871326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:46.878385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:46.878959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:46.879795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.880623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:46.881142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:46.881622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:46.963853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-24560eaf-6721-4466-989c-902004524c6a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.098334 Sep 17 04:35:46.965147 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 385/776] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:35:46 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-interface => generated 285 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:47.090962 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:47.091648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:47.098556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:47.099050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:47.099458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:47.100188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:47.100646 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:47.101058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:47.168532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:47.181549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:47.182534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:47.183233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:47.184928 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:47.186434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-258b0485-bc2d-4437-89cc-274256a6ac9b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1364 microversion: 2.20 time: 0.099436 Sep 17 04:35:47.190169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:47.190944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:47.191809 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:47.192714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:47.193540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:47.194298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:47.201891 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 392/777] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:47 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1364 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:47.448128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:47.456608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a256259e-7b10-4f3c-9a35-c00e4202212b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.280337 Sep 17 04:35:47.471550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 386/778] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:47 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:48.212125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:48.212705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:48.219577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.220073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.220509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.221244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.221699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.222145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.226321 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:48.262387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.262874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.263313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.263763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lazy-loading 'fault' on Instance uuid 0043b5dd-c019-4a76-b2d0-4c68982cac21 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:35:48.304952 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:48.313360 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:48.314830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:48.315996 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-15ea9405-ae6a-4f87-977a-2d193685ad6b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 200 len: 1369 microversion: 2.20 time: 0.108001 Sep 17 04:35:48.316648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 393/779] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:48 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:48.480365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:48.480922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:48.487648 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.488128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.488770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.489747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.490180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.490583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.649484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:48.658188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-130db341-4c43-406f-bd40-a0144c55a357 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.181702 Sep 17 04:35:48.659607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 387/780] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:48 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:48.719390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}, {"name": "network-changed", "server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:48.728199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:48.730622 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:48.731205 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:48.731916 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:48.745447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Creating event network-changed:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:35:48.746012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] Creating event network-changed:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:35:48.751620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-aeeab4ce-36a1-4497-8d2b-23bc5d7c9c84 req-4e4d6db0-358e-424a-88f8-f8d4e7849749 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.036356 Sep 17 04:35:48.752228 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 394/781] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:48 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:49.340455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:49.341283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:49.347922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:49.348676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:49.349127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:49.349832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:49.350291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:49.350713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:49.364359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 HTTP/1.1 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:35:49.366802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:35:49.368372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '0043b5dd-c019-4a76-b2d0-4c68982cac21'} Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:35:49.370240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:35:49.371588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:35:49.374875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:35:49.374875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-82c9b580-a568-4af0-bf61-a736490b4327 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 404 len: 111 microversion: 2.20 time: 0.031865 Sep 17 04:35:49.374875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 388/782] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:49 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:35:49.378185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-029c4caf-e145-4b10-9155-8ac86a4edfcb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '.version_select at 0x7fbc4f43c430>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:49.401799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-029c4caf-e145-4b10-9155-8ac86a4edfcb tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-7217238" status: 204 len: 0 microversion: 2.20 time: 0.028121 Sep 17 04:35:49.402559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 395/783] 10.222.0.60 () {60 vars in 1253 bytes} [Sat Sep 17 04:35:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-7217238 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 17 04:35:49.683601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:49.684353 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:49.690924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:49.691421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:49.691859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:49.692577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:49.693038 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:49.693518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:49.832696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:49.840885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-753382d5-aba8-4a2a-8072-1f680ea313d4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.161287 Sep 17 04:35:49.841823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 389/784] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:49 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:50.421568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-baa2c927-50ce-4127-a6f4-e655dc781593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: .version_select at 0x7fbc4f2fe700>, body: {"keypair": {"name": "tempest-keypair-196185231", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOhYhi7RE8QZQHC+YxXbM1DnTYier8RB+KWdEN0+hXCmDUzg9XNvaaUO5wx/IIWliLkSMfgsMG/6jzW+frDGQlftGdMaQuxpL3h0IhHK7VrNmVN6X4yWpxBD9lJtIa688w=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:50.437624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-baa2c927-50ce-4127-a6f4-e655dc781593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for project e0b194b28dec41119ae0d6d913f19c31. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:35:50.441778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-baa2c927-50ce-4127-a6f4-e655dc781593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for user ed4e695113f843fda52ce62792a36bb0 and project e0b194b28dec41119ae0d6d913f19c31. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:35:50.464743 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-baa2c927-50ce-4127-a6f4-e655dc781593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.047752 Sep 17 04:35:50.465370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 396/785] 10.222.0.60 () {62 vars in 1200 bytes} [Sat Sep 17 04:35:50 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 49 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 17 04:35:51.002995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:51.003613 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:51.009665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:51.010150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:51.010585 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:51.011273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:51.011725 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:51.012157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:51.155569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:51.163532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e719d31c-d580-48ec-8ce5-1fcd8f692761 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.303320 Sep 17 04:35:51.164106 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 390/786] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:50 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:51.329399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "name": "network-vif-plugged", "status": "completed", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:51.340475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:51.341219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:51.341833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:51.354001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] Creating event network-vif-plugged:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:35:51.359144 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-ff4f663b-ff16-4392-96ba-fed8fc6dee16 req-baa5c996-4898-49ca-aa71-f44f53522c9e service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034548 Sep 17 04:35:51.359806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 397/787] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:51.869156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-450347435", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e95cd093-c909-4fa1-980b-df0004dd3cf5"}], "security_groups": [{"name": "tempest-securitygroup--352388421"}], "key_name": "tempest-keypair-196185231", "user_data": "}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:51.978135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] validate_networks() for [('e95cd093-c909-4fa1-980b-df0004dd3cf5', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:35:52.188143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:52.188839 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:52.195015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:52.195727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:52.196158 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:52.196854 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:52.197347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:52.197791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:52.330943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:52.339110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-21091bbb-6c49-4b43-9cb0-48199d8ebc4b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.155645 Sep 17 04:35:52.339644 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 398/788] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:52 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:35:52.456307 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:35:52.456807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:35:52.457476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:35:52.458082 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:35:52.458668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:35:52.472331 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:52.472992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:52.473683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:52.474446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:52.474907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:52.475329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:52.494121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for project e0b194b28dec41119ae0d6d913f19c31. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:35:52.498399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Getting quotas for user ed4e695113f843fda52ce62792a36bb0 and project e0b194b28dec41119ae0d6d913f19c31. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:35:52.509802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:35:52.512153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:35:52.561937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-135db8ea-92d6-4518-892f-d63c16335e75 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.697307 Sep 17 04:35:52.562724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 391/789] 10.222.0.60 () {62 vars in 1188 bytes} [Sat Sep 17 04:35:51 2022] POST /compute/v2.1/servers => generated 401 bytes in 699 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 17 04:35:52.574194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:52.574807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:52.629468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:52.637014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:52.638533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:52.639640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5c927bff-ab2e-4f2a-91db-3f25129f5d4b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1280 microversion: 2.20 time: 0.069546 Sep 17 04:35:52.640193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 399/790] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:52 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1280 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:53.367326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:53.368367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:53.379653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:53.380445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:53.381167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:53.382380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:53.383127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:53.383823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:53.519187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:53.527228 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9fc4dcbf-eb64-455d-91a2-25c5577b3a8f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1265 microversion: 2.1 time: 0.167481 Sep 17 04:35:53.527838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 392/791] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:53 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1265 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:53.664851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:53.666284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:53.674262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:53.674750 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:53.675202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:53.675893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:53.676336 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:53.676748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:53.738534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:53.747284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:53.748704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:53.749830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bf14b076-a791-41e8-99f6-48494c2c5db3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1276 microversion: 2.20 time: 0.089665 Sep 17 04:35:53.750411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 400/792] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:53 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1276 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:54.550315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:54.551068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:54.558279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.558839 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.559372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.560393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.560962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.561458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.704516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:54.712713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f4fb6ae3-97df-4c00-849e-c80c025501c8 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1453 microversion: 2.1 time: 0.166465 Sep 17 04:35:54.713333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 393/793] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:54 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1453 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:54.735653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:54.736308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:54.743827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.744295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.744759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.745499 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.745935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.746382 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.777271 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:54.777895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:54.784080 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.784608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.785053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.785826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:54.786283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:54.786706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:54.790972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:35:54.852124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:54.860678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:54.862221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:54.863412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-46e748f9-5a66-4f0a-9fc9-9cac5d0b2cf6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1338 microversion: 2.20 time: 0.090303 Sep 17 04:35:54.863953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 394/794] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:54 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1338 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:54.902429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:54.910377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a60fcaa6-ea50-446f-b2be-73e72f5d90a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1453 microversion: 2.1 time: 0.178680 Sep 17 04:35:54.910942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 401/795] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:35:54 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1453 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:35:55.887940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:55.888670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:55.896582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:55.898166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:55.898683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:55.899659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:55.900310 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:55.900930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:56.042173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:56.050725 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:56.052274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:56.053517 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d27e1f11-1a43-48ad-86b8-2d42a2745d40 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.169542 Sep 17 04:35:56.054150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 395/796] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:55 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:57.078799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:57.079709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:57.086522 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:57.087333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:57.087944 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:57.088832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:57.089312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:57.089855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:57.423268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:57.432198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:57.434052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:57.435244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c883b9ff-4097-4b86-8199-1a0c0545af9e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.360566 Sep 17 04:35:57.435808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 402/797] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:57 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:57.550914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:57.563538 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:57.564038 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:57.564721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:57.576063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] Creating event network-changed:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:35:57.580988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-229588d0-aa46-40fc-a759-a9dfd6a72683 req-b6970cd5-2d25-43f2-8957-2d1f02195eb9 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034927 Sep 17 04:35:57.581622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 396/798] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:58.461507 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:58.462135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:58.469127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:58.469726 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:58.470181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:58.470876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:58.471323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:58.471744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:58.611181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:58.619951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:58.621426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:58.622551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0c50dc2e-e560-4c4d-92f9-1365e79610f0 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.165039 Sep 17 04:35:58.623086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 403/799] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:58 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:35:59.614358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:35:59.624773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:59.625315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:59.625771 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:59.636728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] Creating event network-changed:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:35:59.641823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-39112b94-56de-40fe-8bc9-29c2fcce48a6 req-1182ea39-2001-4bbf-bc8a-0b4bf41019e7 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031769 Sep 17 04:35:59.642659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 397/800] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:35:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:35:59.648700 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:35:59.649402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:35:59.658430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:59.658922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:59.659361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:59.660058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:35:59.660519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:35:59.660945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:35:59.795371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:59.807918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:59.810236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:35:59.811965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c54c6dec-c565-424c-a07f-187473bab4ca tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.168225 Sep 17 04:35:59.812829 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 404/801] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:35:59 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:00.837484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:00.838179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:00.845720 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:00.846225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:00.846664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:00.847369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:00.847838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:00.848258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:01.319675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:01.328292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:01.329799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:01.330951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-226f3530-752f-4153-8b4e-0025ada7773d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.498505 Sep 17 04:36:01.331558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 398/802] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:36:00 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:02.360238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:02.361010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:02.369500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:02.370098 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:02.370635 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:02.371446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:02.372045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:02.372566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:02.507605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:02.515911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:02.517444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:02.518562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e618c62f-e5fa-4bf4-9bfc-3ce044021555 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.165098 Sep 17 04:36:02.519127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 405/803] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:36:02 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:03.433114 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "name": "network-vif-plugged", "status": "completed", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:36:03.438944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:03.439749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:03.440655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:03.449153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:03.449639 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:03.450066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:03.463203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] Creating event network-vif-plugged:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:36:03.468455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-a7f7114e-06aa-46f4-b518-f13e01bad470 req-5b124a00-68ce-4118-87af-5ff6dc45521f service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038674 Sep 17 04:36:03.469108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 399/804] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:36:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:36:03.543822 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:03.544461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:03.546528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:03.552626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:03.553119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:03.553611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:03.554300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:03.554771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:03.555201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:03.691183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:03.699785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:03.701328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:03.702509 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1ddc8472-2a08-4dac-9226-67fb5c8f7d2a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.162903 Sep 17 04:36:03.703053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 406/805] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:36:03 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:04.732870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:04.734034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:04.741526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:04.742007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:04.742435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:04.743119 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:04.743563 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:04.743984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:04.903847 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:04.917303 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:04.919679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:04.921490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ae5b3ab8-78d3-4196-a624-5022971eb2ea tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1405 microversion: 2.20 time: 0.195820 Sep 17 04:36:04.922318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 400/806] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:36:04 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1405 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:05.946487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:05.947176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:05.953618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:05.954111 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:05.954548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:05.955238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:05.955692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:05.956117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:06.085776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:06.094420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:06.095833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:06.096993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-523cd9c2-4dd8-4045-86ee-3e14f483e7be tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1596 microversion: 2.20 time: 0.154712 Sep 17 04:36:06.097612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 407/807] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:36:05 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1596 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:36:06.121581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:06.122107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:06.128961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:06.129487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:06.129959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:06.130653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:06.131110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:06.131544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:06.210445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-14d82089-8769-4984-b67c-62de5e1d5369 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.092807 Sep 17 04:36:06.211052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 401/808] 10.222.0.60 () {60 vars in 1316 bytes} [Sat Sep 17 04:36:06 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:36:07.188318 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:36:07.198860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:07.199441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:07.199938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:07.211787 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] Creating event network-changed:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:36:07.216997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-7e715056-9c51-4b0d-af04-bb45bcf5b57a req-f543120c-f7f1-4ae0-a53b-07a79ac6e30f service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032389 Sep 17 04:36:07.217642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 408/809] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:36:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:36:52.577309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=5 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:52.578068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=32 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:52.582545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:52.583046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:52.589983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:52.590486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:52.590903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:52.591599 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:52.592424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:52.592852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:52.675768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'services' on Instance uuid 260f958f-caf1-46d9-8a2e-74816fa1bfd6 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:36:52.754179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0ba7761f-ebae-4942-a70a-cdd968e8c9da tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments/bbbe9e0a-0e35-4feb-a19d-5f4724115867" status: 202 len: 0 microversion: 2.1 time: 0.178243 Sep 17 04:36:52.754734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 402/810] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:36:52 2022] DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments/bbbe9e0a-0e35-4feb-a19d-5f4724115867 => generated 0 bytes in 180 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:36:52.761063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=40,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:52.761859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=66 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:36:52.767010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:52.769577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:52.778727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:52.779288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:52.779832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:52.780523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:52.780971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:52.781447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:52.814066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a94563f4-4c2f-4907-b421-c19a07385e3e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054644 Sep 17 04:36:52.814621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 409/811] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:36:52 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:36:53.829237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:53.831766 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:53.838294 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:53.838793 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:53.839237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:53.839942 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:53.840403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:53.840837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:53.874258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3bca3e82-29f5-4d9f-92b3-ca59ccd0dea2 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051009 Sep 17 04:36:53.874834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 403/812] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:36:53 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:36:53.953991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:53.954574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:53.961055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:53.961567 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:53.961977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:53.962651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:53.963097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:53.963817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:53.989114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:36:54.077201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-61a6f6da-8eab-434a-807e-87e2a300f3a0 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 204 len: 0 microversion: 2.1 time: 0.126692 Sep 17 04:36:54.078429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 410/813] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:36:53 2022] DELETE /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:36:54.086176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:54.086816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:54.093479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:54.093956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:54.094384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:54.095059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:54.095507 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:54.095922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:54.240633 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:54.252356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-050b1c19-f431-438f-8601-1063da42e22f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1689 microversion: 2.1 time: 0.169863 Sep 17 04:36:54.253108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 404/814] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:36:54 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1689 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:36:55.281331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:55.282443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:55.293355 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:55.294159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:55.294883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:55.296012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:55.297071 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:55.297878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:55.440976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:55.449197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-22979fe0-bb47-494a-b473-f571e0fa8497 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1689 microversion: 2.1 time: 0.175109 Sep 17 04:36:55.449867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 411/815] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:36:55 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1689 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:36:56.476320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:56.476910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:56.483908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:56.484402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:56.484849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:56.485575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:56.486032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:56.486460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:56.615016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:56.623098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-76df922a-c371-4093-9076-886ffa2f49a4 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1689 microversion: 2.1 time: 0.152065 Sep 17 04:36:56.623679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 405/816] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:36:56 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1689 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:36:57.647070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:57.647656 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:57.654676 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:57.655159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:57.655626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:57.656315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:57.656768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:57.657255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:57.787723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:57.795906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4936d8a7-60b9-417d-a011-199714a76c4f tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1689 microversion: 2.1 time: 0.152879 Sep 17 04:36:57.796476 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 412/817] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:36:57 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1689 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:36:58.820115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:36:58.821769 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:36:58.829845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:58.830762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:58.831587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:58.833272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:58.834069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:58.834723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:59.008649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:36:59.020280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "260f958f-caf1-46d9-8a2e-74816fa1bfd6", "name": "network-vif-deleted", "tag": "ace73665-81e0-4dd4-bd89-8ca9daa484ad"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:36:59.024688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9680c48e-52d6-4079-ab10-b9706d551737 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1622 microversion: 2.1 time: 0.209134 Sep 17 04:36:59.025607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 406/818] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:36:58 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1622 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:36:59.030367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:36:59.030792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:36:59.031236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:36:59.042722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] Creating event network-vif-deleted:ace73665-81e0-4dd4-bd89-8ca9daa484ad for instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 on hv-cinder-85799 Sep 17 04:36:59.047709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-48deff52-da74-493e-805e-cbb817ca909a req-d08a6615-fc20-49b8-8d64-3341b1312210 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031274 Sep 17 04:36:59.049419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 413/819] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:36:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:37:00.048339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:00.048957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:00.055665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:00.056158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:00.056619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:00.057344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:00.057801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:00.058223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:00.091948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:00.092443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:00.092889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:00.093381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'fault' on Instance uuid 260f958f-caf1-46d9-8a2e-74816fa1bfd6 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:37:00.135262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:00.142935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6ed9a778-55c7-4ea5-97a4-defdf80771fe tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 200 len: 1336 microversion: 2.1 time: 0.098648 Sep 17 04:37:00.143504 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 407/820] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:00 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 1336 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:01.168230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:01.168960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 260f958f-caf1-46d9-8a2e-74816fa1bfd6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:01.176024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:01.176534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:01.176997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:01.177895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:01.178357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:01.178783 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:01.192584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] HTTP exception thrown: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Request method failure captured: Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 HTTP/1.1 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU1VyO3YwjgPsO3zAE4FZmsyWEcgb5B0Oy5RcSv7jAJnr5DJfg3c80tKcUCh5_S8hLjEcJ5qglAE8bqRQMN3PZ86lwmiGwGAxYPEQcqx_A0AYCoaReV-dg5nm3Mg-oVytdBrAl3CriJDBmv83kbBOhg2wmCsC9XwgpE4vLYHXFOTWwg Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:37:01.194348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 5bb515269d1a45f594f3d297c7fd267e Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. Sep 17 04:37:01.196073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '260f958f-caf1-46d9-8a2e-74816fa1bfd6'} Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:01.196988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. Sep 17 04:37:01.198519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:01.200046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Returning 404 to user: Instance 260f958f-caf1-46d9-8a2e-74816fa1bfd6 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:01.200046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-530214d5-44c8-4b17-b468-de383d96a8e3 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6" status: 404 len: 111 microversion: 2.1 time: 0.033221 Sep 17 04:37:01.200046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 414/821] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:01 2022] GET /compute/v2.1/servers/260f958f-caf1-46d9-8a2e-74816fa1bfd6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:37:01.204731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e86817f-b51b-413c-8112-3ecd3f5a10dd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '.version_select at 0x7fbc4f45e040>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:01.226142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e86817f-b51b-413c-8112-3ecd3f5a10dd tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1018766987" status: 202 len: 0 microversion: 2.1 time: 0.025392 Sep 17 04:37:01.226678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 408/822] 10.222.0.60 () {58 vars in 1221 bytes} [Sat Sep 17 04:37:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1018766987 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:37:04.143398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:04.144045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:04.151844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:04.152329 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:04.152763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:04.153501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:04.153956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:04.154382 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:04.180090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:37:04.280255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c696a63a-6335-495a-96f7-dd3b94947728 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 204 len: 0 microversion: 2.1 time: 0.140869 Sep 17 04:37:04.281231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 415/823] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:37:04 2022] DELETE /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:37:04.292561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:04.293135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:04.304344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:04.305021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:04.305746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:04.306446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:04.306904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:04.307331 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:04.442353 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:04.450823 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-23800d60-2ef6-4c9c-9e7e-32afbab62e7e tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1508 microversion: 2.1 time: 0.165277 Sep 17 04:37:04.451482 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 409/824] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:04 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1508 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:37:05.474240 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:05.474951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:05.481718 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:05.482204 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:05.482636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:05.483331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:05.483800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:05.484225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:05.620296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:05.628302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d2311915-986e-4d09-aa7e-b10a00c66975 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 200 len: 1508 microversion: 2.1 time: 0.158328 Sep 17 04:37:05.628885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 416/825] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:05 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 1508 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:05.853871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "33c2f844-d503-44e7-86dd-b67034f5ca40", "name": "network-vif-deleted", "tag": "4de14b41-d3ca-42f2-b3cd-1e608f2d4c84"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:05.865135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:05.865607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:05.866314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:05.878650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] Creating event network-vif-deleted:4de14b41-d3ca-42f2-b3cd-1e608f2d4c84 for instance 33c2f844-d503-44e7-86dd-b67034f5ca40 on hv-cinder-85799 Sep 17 04:37:05.884705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-43a45afd-4e3d-4ae6-aad8-95e9ffc5c2c2 req-c1b76298-6f1b-483f-a40b-521ca30266b0 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034600 Sep 17 04:37:05.885410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 410/826] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:06.653812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:06.654633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: 33c2f844-d503-44e7-86dd-b67034f5ca40] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:06.661910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.662495 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.662928 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.663814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.664262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.664682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.678328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] HTTP exception thrown: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Request method failure captured: Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 HTTP/1.1 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU1VyO3YwjgPsO3zAE4FZmsyWEcgb5B0Oy5RcSv7jAJnr5DJfg3c80tKcUCh5_S8hLjEcJ5qglAE8bqRQMN3PZ86lwmiGwGAxYPEQcqx_A0AYCoaReV-dg5nm3Mg-oVytdBrAl3CriJDBmv83kbBOhg2wmCsC9XwgpE4vLYHXFOTWwg Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:37:06.679978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 5bb515269d1a45f594f3d297c7fd267e Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. Sep 17 04:37:06.681663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '33c2f844-d503-44e7-86dd-b67034f5ca40'} Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:06.682811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. Sep 17 04:37:06.684132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:06.685619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Returning 404 to user: Instance 33c2f844-d503-44e7-86dd-b67034f5ca40 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:06.685619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-50277077-aaec-4f8b-b52c-ae9eec0cafce tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40" status: 404 len: 111 microversion: 2.1 time: 0.033646 Sep 17 04:37:06.685619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 417/827] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:06 2022] GET /compute/v2.1/servers/33c2f844-d503-44e7-86dd-b67034f5ca40 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 17 04:37:06.691283 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:06.691881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:06.698242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.698726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.699165 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.699850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.700310 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.700736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.727508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:37:06.810972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a06da057-2db6-4a63-bc69-3ee20ead9298 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "DELETE /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 204 len: 0 microversion: 2.1 time: 0.123844 Sep 17 04:37:06.811773 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 411/828] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:37:06 2022] DELETE /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 17 04:37:06.821240 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:06.821857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:06.828282 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.828772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.829258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.829954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:06.830434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:06.830860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:06.967503 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:06.979660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7d3468e2-0f56-44b8-b6ee-e21f7046660a tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.163301 Sep 17 04:37:06.980230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 418/829] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:06 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:08.008139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:08.009071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:08.020031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:08.020832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:08.021616 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:08.022756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:08.023521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:08.024228 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:08.166540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:08.174874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f4d424f7-1fcc-47c5-8ce5-1d36ed39571b tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.174211 Sep 17 04:37:08.175443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 412/830] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:07 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:09.198805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:09.199429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:09.206465 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:09.206970 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:09.207385 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:09.208195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:09.208643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:09.209065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:09.354140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:09.365427 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-72c17fdf-ea4f-43d0-887f-b8dad7820285 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1521 microversion: 2.1 time: 0.170370 Sep 17 04:37:09.366280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 419/831] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:09 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:09.928787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c0837134-994e-4402-a813-3bacee6f4853", "name": "network-vif-deleted", "tag": "55d9eabb-ceb8-459c-a4ad-0c9da0df432e"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:09.941163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:09.941646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:09.942074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:09.953879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] Creating event network-vif-deleted:55d9eabb-ceb8-459c-a4ad-0c9da0df432e for instance c0837134-994e-4402-a813-3bacee6f4853 on hv-cinder-85799 Sep 17 04:37:09.959468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-e835aa84-96c0-4c8f-9ea2-13dd588d3f74 req-c026477f-2e03-4e98-9ce3-ccc1e6a8c401 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034313 Sep 17 04:37:09.960338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 413/832] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:37:10.113152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:10.113582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:10.120280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:10.121202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:10.121202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:10.122028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:10.122480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:10.122902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:10.146836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6049209b-657a-4abc-b320-7ceae1f6fd88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/ips" status: 200 len: 149 microversion: 2.1 time: 0.037641 Sep 17 04:37:10.147401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 420/833] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:37:10 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/ips => generated 149 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:10.390403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:10.391453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:10.397939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:10.398532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:10.399158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:10.400069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:10.400732 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:10.401383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:10.436729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:10.437400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:10.437945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:10.438573 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lazy-loading 'fault' on Instance uuid c0837134-994e-4402-a813-3bacee6f4853 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:37:10.481022 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5bb515269d1a45f594f3d297c7fd267e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd54f5f66e0eb4be9aad2e6d37e0b6db6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:10.488775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ba755892-67c7-43d3-a12b-d5403e08fa45 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 200 len: 1312 microversion: 2.1 time: 0.103686 Sep 17 04:37:10.489376 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 414/834] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:10 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 1312 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:11.506826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d7691dc8-69f2-48bf-bf74-c3aa9215f319"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:11.509320 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:11.511831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:11.512477 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] [instance: c0837134-994e-4402-a813-3bacee6f4853] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:11.516445 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:11.516938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:11.517410 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:11.518118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:11.518577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:11.518998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:11.519253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:11.519726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:11.520158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:11.520848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:11.521530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:11.521961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:11.535681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] HTTP exception thrown: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Request method failure captured: Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 HTTP/1.1 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU1VyO3YwjgPsO3zAE4FZmsyWEcgb5B0Oy5RcSv7jAJnr5DJfg3c80tKcUCh5_S8hLjEcJ5qglAE8bqRQMN3PZ86lwmiGwGAxYPEQcqx_A0AYCoaReV-dg5nm3Mg-oVytdBrAl3CriJDBmv83kbBOhg2wmCsC9XwgpE4vLYHXFOTWwg Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:37:11.537383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/d54f5f66e0eb4be9aad2e6d37e0b6db6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: d54f5f66e0eb4be9aad2e6d37e0b6db6 Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-528203577 Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 5bb515269d1a45f594f3d297c7fd267e Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-ServerRescueNegativeTestJSON-528203577-project Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. Sep 17 04:37:11.538912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'c0837134-994e-4402-a813-3bacee6f4853'} Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:11.539965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. Sep 17 04:37:11.541527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:11.542899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] Returning 404 to user: Instance c0837134-994e-4402-a813-3bacee6f4853 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:11.542899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-88293fef-2e06-4f07-9d37-790a35cb3033 tempest-ServerRescueNegativeTestJSON-528203577 tempest-ServerRescueNegativeTestJSON-528203577-project] 10.222.0.60 "GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853" status: 404 len: 111 microversion: 2.1 time: 0.030922 Sep 17 04:37:11.542899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 415/835] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:11 2022] GET /compute/v2.1/servers/c0837134-994e-4402-a813-3bacee6f4853 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:37:11.792903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f1d81e90-ab4a-494b-9e7f-6e9a70c3855b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.289940 Sep 17 04:37:11.793701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 421/836] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:37:11 2022] POST /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments => generated 194 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:16.252699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-035eb7bc-09e2-49c7-b8fa-5edf9ada5a49 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Action: 'create', calling method: .version_select at 0x7fbc4f41ab80>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-433443288", "metadata": {"Type": "work"}}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:16.560273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-035eb7bc-09e2-49c7-b8fa-5edf9ada5a49 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.462370 Sep 17 04:37:16.560911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 416/837] 10.222.0.60 () {60 vars in 1156 bytes} [Sat Sep 17 04:37:16 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 464 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 17 04:37:16.575349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bf492b2f-7eeb-4255-8025-37eb938e9779 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f2caa60>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:16.626000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bf492b2f-7eeb-4255-8025-37eb938e9779 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 200 len: 352 microversion: 2.1 time: 0.055549 Sep 17 04:37:16.626577 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 422/838] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:37:16 2022] GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 352 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:17.641745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8a6a5d8d-fafc-4c88-b397-37a8468f4279 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f4263a0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:17.692116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8a6a5d8d-fafc-4c88-b397-37a8468f4279 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 200 len: 353 microversion: 2.1 time: 0.055168 Sep 17 04:37:17.692688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 417/839] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:37:17 2022] GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 353 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:17.706251 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b6786e2f-ed69-48a2-9cf8-5d025ac34df0 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f482a60>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:17.754424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b6786e2f-ed69-48a2-9cf8-5d025ac34df0 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 200 len: 353 microversion: 2.1 time: 0.052496 Sep 17 04:37:17.754997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 423/840] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:37:17 2022] GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:17.778697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f73de48c-78a4-4a43-85a4-a70fbcc5136a tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f42faf0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:17.840651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f73de48c-78a4-4a43-85a4-a70fbcc5136a tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 202 len: 0 microversion: 2.1 time: 0.065731 Sep 17 04:37:17.841834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 418/841] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:37:17 2022] DELETE /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:37:17.850101 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ee0f7f13-b609-40b4-8b36-7934fd7d9c4e tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f43caf0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:17.900777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ee0f7f13-b609-40b4-8b36-7934fd7d9c4e tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 200 len: 352 microversion: 2.1 time: 0.054743 Sep 17 04:37:17.901437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 424/842] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:37:17 2022] GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 352 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:18.920332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f1ae6845-f90c-4b7a-912c-e5b72b4ef0c8 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Calling method '.version_select at 0x7fbc4f4834c0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:18.952642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-f1ae6845-f90c-4b7a-912c-e5b72b4ef0c8 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] HTTP exception thrown: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f1ae6845-f90c-4b7a-912c-e5b72b4ef0c8 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Request method failure captured: Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 HTTP/1.1 Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU78_SS4WPm5HmPv5rNMyeUr7d-dAizrtAfJHB2CISMookJ0hsF9ftY3qfIlM4JY7zpBdtSjoT-Z0njmVw2eDQ2MA0-Xy5ZRw8Ithb3G3zD-2X4v1KauITErfh-M_FUzL2vF1puBc2z8IixIyIvn0DPg3CHAR0_VDS75SChuXkdvVKg Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 928e619eeefb45bf8e8597df083b1fce Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesGetTestJSON-192509252 Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:37:18.954623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/928e619eeefb45bf8e8597df083b1fce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/928e619eeefb45bf8e8597df083b1fce"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_928e619eeefb45bf8e8597df083b1fce"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/928e619eeefb45bf8e8597df083b1fce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesGetTestJSON-192509252 Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 928e619eeefb45bf8e8597df083b1fce Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesGetTestJSON-192509252 Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesGetTestJSON-192509252-project Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 9af292fc356c4276b70d718089051eb2 Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesGetTestJSON-192509252-project Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '5ed670b6-6736-4845-a9f8-b32520b6c1d5'} Sep 17 04:37:18.956259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. (HTTP 404) (Request-ID: req-4516e8b2-3ba4-43d5-8641-e267c35a362d) Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:18.957386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:37:18.958871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 17 04:37:18.960178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f1ae6845-f90c-4b7a-912c-e5b72b4ef0c8 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] Returning 404 to user: Volume 5ed670b6-6736-4845-a9f8-b32520b6c1d5 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f1ae6845-f90c-4b7a-912c-e5b72b4ef0c8 tempest-VolumesGetTestJSON-192509252 tempest-VolumesGetTestJSON-192509252-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5" status: 404 len: 109 microversion: 2.1 time: 0.042694 Sep 17 04:37:18.961706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 419/843] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:37:18 2022] GET /compute/v2.1/os-volumes/5ed670b6-6736-4845-a9f8-b32520b6c1d5 => generated 109 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:37:19.385977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d7691dc8-69f2-48bf-bf74-c3aa9215f319"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:19.388789 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:19.395511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.396027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.396458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.397202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.397648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.398072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.485406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: POST /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments HTTP/1.1 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 74 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:37:19.487236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {"volumeAttachment": {"volumeId": "d7691dc8-69f2-48bf-bf74-c3aa9215f319"}} Sep 17 04:37:19.488833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'server_id': '8bc28c77-6b66-4f4e-819e-c6047e0e6ddd', 'body': {'volumeAttachment': {'volumeId': 'd7691dc8-69f2-48bf-bf74-c3aa9215f319'}}} Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:19.489874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:19.491149 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 400 to user: Invalid volume: volume d7691dc8-69f2-48bf-bf74-c3aa9215f319 is already attached to instances: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dc6f01f7-38d5-419e-a107-f0027c941ebe tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.107933 Sep 17 04:37:19.492326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 425/844] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:37:19 2022] POST /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments => generated 174 bytes in 109 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:37:19.557749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:19.558242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:19.564439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.564925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.565393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.566075 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.566526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.566983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.653110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'services' on Instance uuid 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:37:19.730574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9cb51c38-60cc-4ca4-b712-18f37f70d57f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments/d7691dc8-69f2-48bf-bf74-c3aa9215f319" status: 202 len: 0 microversion: 2.1 time: 0.177107 Sep 17 04:37:19.731130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 420/845] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:37:19 2022] DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments/d7691dc8-69f2-48bf-bf74-c3aa9215f319 => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:37:19.739754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:19.741949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:19.749865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.750395 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.750790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.751670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:19.752119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:19.752532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:19.756934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:19.785299 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5c088634-747b-4520-a93f-8737e3572514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049526 Sep 17 04:37:19.785871 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 426/846] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:37:19 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:37:20.798883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:20.801282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:20.809354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:20.809880 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:20.810328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:20.811036 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:20.811898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:20.812347 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:20.839879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:20.845625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dc54e74a-eb75-4888-9c09-094d5ac78891 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051163 Sep 17 04:37:20.846304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 421/847] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:37:20 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:37:20.910131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:20.910971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:20.917870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:20.918606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:20.919114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:20.919909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:20.920429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:20.920954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:20.946197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:37:21.024797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e532f50d-2431-4417-aa01-786fc37b7ad6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 204 len: 0 microversion: 2.1 time: 0.119136 Sep 17 04:37:21.025384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 427/848] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:37:20 2022] DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 17 04:37:21.034217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:21.034937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:21.045113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:21.045674 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:21.046450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:21.047231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:21.047791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:21.048280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:21.180329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:21.188421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0ad10cf9-a553-4940-84ba-114ab4645fe6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1679 microversion: 2.1 time: 0.158813 Sep 17 04:37:21.189149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 422/849] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:21 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1679 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:22.217048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:22.217840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:22.224657 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:22.225185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:22.225625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:22.226320 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:22.226831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:22.227193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:22.371375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:22.379252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f0833a16-0c1e-4e2d-bd4f-13c781bc604a tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1679 microversion: 2.1 time: 0.169139 Sep 17 04:37:22.379783 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 428/850] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:22 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1679 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:23.403198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:23.403790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:23.405676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=45,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:23.410813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:23.411297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:23.411726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:23.412425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:23.412868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:23.413300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:23.651955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:23.660239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-70e0de47-0906-4025-9ec2-b5c32113d9a5 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1679 microversion: 2.1 time: 0.261164 Sep 17 04:37:23.660746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 423/851] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:23 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1679 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:24.037292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8bc28c77-6b66-4f4e-819e-c6047e0e6ddd", "name": "network-vif-deleted", "tag": "fa109493-7edb-4c37-bc35-5135e4fef636"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:24.043398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:24.051174 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:24.051780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:24.052426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:24.066082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] Creating event network-vif-deleted:fa109493-7edb-4c37-bc35-5135e4fef636 for instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd on hv-cinder-85799 Sep 17 04:37:24.076086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-b24c407a-0710-48b2-8fd8-b202768955c3 req-9791b166-3132-4a61-8576-e859d9d0efc5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043244 Sep 17 04:37:24.077113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 429/852] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:24.684149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:24.685653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:24.693364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:24.694004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:24.694546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:24.695327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:24.696015 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:24.696445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:24.735387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:24.735870 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:24.736484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:24.736943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'fault' on Instance uuid 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:37:24.784218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:24.792827 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3af16598-824a-46ed-b93b-970ad1f56988 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 200 len: 1331 microversion: 2.1 time: 0.112644 Sep 17 04:37:24.793439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 424/853] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:24 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 1331 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:25.196487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:25.197027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:25.204235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:25.204740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:25.205252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:25.205921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:25.206412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:25.206845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:25.231046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c4b1e31b-c1dd-4ae8-a552-45792d2372ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/ips" status: 200 len: 150 microversion: 2.20 time: 0.039403 Sep 17 04:37:25.236224 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 430/854] 10.222.0.60 () {60 vars in 1289 bytes} [Sat Sep 17 04:37:25 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:37:25.815986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:25.816631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:25.823189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:25.823682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:25.824121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:25.824819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:25.825345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:25.826347 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:25.840989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd HTTP/1.1 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:37:25.843029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:37:25.844417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '8bc28c77-6b66-4f4e-819e-c6047e0e6ddd'} Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:37:25.845446 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:37:25.847200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:37:25.849117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:37:25.849117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-57afbb04-8c00-43b3-9e8c-e2613eabd039 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 404 len: 111 microversion: 2.1 time: 0.032381 Sep 17 04:37:25.849117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 425/855] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:25 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:37:25.853545 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aae9b67f-cde7-4488-b796-820ecead957f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '.version_select at 0x7fbc4f482310>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:25.873286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aae9b67f-cde7-4488-b796-820ecead957f tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1441753317" status: 202 len: 0 microversion: 2.1 time: 0.023500 Sep 17 04:37:25.873870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 431/856] 10.222.0.60 () {58 vars in 1221 bytes} [Sat Sep 17 04:37:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1441753317 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:37:26.624005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-67306697-ed21-46e3-8815-48d4d23c2514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: .version_select at 0x7fbc4f4a7280>, body: {"keypair": {"name": "tempest-keypair-2006014554", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK97kCs5s7ElM6KFEmDJ1qCm8TG1zFzThVxtk8n20bbVrOQwfl5ypFCbMygj65iQCCXze+/siWZgxEI1I+5OhacySiZWb6NTWS+iiYlK92/2x1R2ZKodfAuFubsVgMdwmw=="}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:26.640601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-67306697-ed21-46e3-8815-48d4d23c2514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:37:26.644540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-67306697-ed21-46e3-8815-48d4d23c2514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:37:26.675100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-67306697-ed21-46e3-8815-48d4d23c2514 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.057222 Sep 17 04:37:26.675706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 426/857] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:37:26 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:27.143151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:27.143684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:27.153150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:27.153898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:27.154518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:27.156059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:27.156517 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:27.156947 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:27.182776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lazy-loading 'flavor' on Instance uuid ad101a32-7ee9-4928-b7e8-2ea39a74bf6e {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:37:27.406826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3b0ef33e-a850-4713-b9ca-54a2c5c87468 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/action" status: 202 len: 0 microversion: 2.20 time: 0.268835 Sep 17 04:37:27.411233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 432/858] 10.222.0.60 () {62 vars in 1319 bytes} [Sat Sep 17 04:37:27 2022] POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/action => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:37:27.418992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:27.420626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:27.430799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:27.431394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:27.432312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:27.432708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:27.433319 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:27.433842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:27.639238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:27.650873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:27.652572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:27.654105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-41cc1311-52e1-45e6-8f4c-96ecfaa27e71 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.241351 Sep 17 04:37:27.654662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 427/859] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:27 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:28.136211 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1157107057", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5d741fbf-1171-42a3-8de2-aef8bbcc1581"}], "security_groups": [{"name": "tempest-securitygroup--2118951385"}], "key_name": "tempest-keypair-2006014554", "user_data": "}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:28.248787 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] validate_networks() for [('5d741fbf-1171-42a3-8de2-aef8bbcc1581', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:37:28.629333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:37:28.629778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:37:28.630248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:37:28.630655 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:37:28.631087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:37:28.640759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:28.641284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:28.641779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:28.642501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:28.642958 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:28.643430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:28.665836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:37:28.671865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Getting quotas for user 094abcdcd1f54a658564357056e7849a and project e9f593f419bb4ebeb351a9ad54a2a532. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:37:28.680302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:28.680912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:28.688685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:28.689345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:28.689887 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:28.690689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:28.691694 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:28.692223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:28.692628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:37:28.695095 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:37:28.744278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-87edebac-a512-4dcc-823e-8b4f40ff0812 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.612351 Sep 17 04:37:28.744937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 433/860] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:37:28 2022] POST /compute/v2.1/servers => generated 402 bytes in 614 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:37:28.756222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-46da1a67-73df-40d8-8e53-198bcce14929 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:28.756837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-46da1a67-73df-40d8-8e53-198bcce14929 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:28.828689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-46da1a67-73df-40d8-8e53-198bcce14929 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:28.836074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-46da1a67-73df-40d8-8e53-198bcce14929 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1242 microversion: 2.1 time: 0.083665 Sep 17 04:37:28.849330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 434/861] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:28 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1242 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:37:28.870381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:28.881203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:28.882632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:28.883802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4458d1c1-5e9c-46ca-b944-1b316690115c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.207373 Sep 17 04:37:28.893191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 428/862] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:28 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:37:29.864918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:29.865512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:29.873039 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:29.873889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:29.874048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:29.874684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:29.875203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:29.875564 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:29.910352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:29.911055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:29.918182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:29.918654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:29.919310 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:29.920125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:29.920560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:29.921172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:29.945515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:29.953705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e4398f4a-f7ab-4357-aad2-5b2aebb24791 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1294 microversion: 2.1 time: 0.093213 Sep 17 04:37:29.954444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 435/863] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:29 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1294 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:30.142425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:30.153638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:30.155146 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:30.156758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-942fc458-63a5-4427-9d38-42c08db1dcc9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.250329 Sep 17 04:37:30.156853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 429/864] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:29 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:37:30.977904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:30.978534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:30.984994 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:30.985505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:30.985936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:30.986632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:30.987080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:30.987500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:31.118785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:31.126853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5ee57ce2-b310-4f55-969e-9263082cd716 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.152860 Sep 17 04:37:31.127397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 436/865] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:30 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:31.181335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:31.181971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:31.188497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:31.188986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:31.189460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:31.190213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:31.190613 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:31.191041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:31.682109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:31.692479 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:31.694234 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:31.696020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-387c6d6b-29e4-4b5b-94d9-3af4307e8d7f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.518333 Sep 17 04:37:31.696799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 430/866] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:31 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:31.882062 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "871c4641-f974-4cc5-bc3e-0c87446fedc6", "tag": "b2b856a4-73a0-498f-8867-d7261d331366"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:31.893127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:31.895988 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:31.897056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:31.909333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] Creating event network-changed:b2b856a4-73a0-498f-8867-d7261d331366 for instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 on hv-cinder-85799 Sep 17 04:37:31.914534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-399ffdce-378f-4ca8-83fe-b8dcba5ae239 req-fa035413-5f28-43b6-8c63-fc40c7bab7d7 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036691 Sep 17 04:37:31.915324 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 437/867] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:37:32.153310 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:32.154028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:32.161048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:32.161615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:32.162053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:32.162923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:32.163590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:32.163955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:32.415460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:32.423603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ea027814-9a7c-4e1c-9a29-4063fe5488cf tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.275983 Sep 17 04:37:32.424149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 431/868] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:32 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:32.722995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:32.723759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:32.730316 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:32.730819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:32.731252 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:32.731983 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:32.732441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:32.732868 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:32.877896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:32.887692 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:32.889668 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:32.890855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-41736fed-8c51-441e-af3a-28b0b2b92e72 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.172644 Sep 17 04:37:32.891419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 438/869] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:32 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:33.447380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:33.447999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:33.454670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:33.455152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:33.455588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:33.456286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:33.456729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:33.457426 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:33.634866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:33.643157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-38754326-84ac-49c6-b1c2-0d9a93b8179d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.199766 Sep 17 04:37:33.643635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 432/870] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:33 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:33.916075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:33.916783 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:33.923455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:33.923931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:33.924357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:33.925037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:33.925508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:33.925924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:34.062470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:34.070760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:34.072193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:34.073387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6eebdb8e-5cb7-4ae8-9d33-6f0b15aff276 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.161249 Sep 17 04:37:34.073924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 439/871] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:33 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:34.668312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:34.668312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:34.674669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:34.675341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:34.675867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:34.676608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:34.677309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:34.677730 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:34.897796 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:34.907018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-deb8b301-5faa-4ffb-920b-33a9b406ecce tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.245059 Sep 17 04:37:34.921488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 433/872] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:34 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:35.099863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:35.101431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:35.108757 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:35.109516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:35.110244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:35.111353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:35.111838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:35.112482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:35.268331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:35.278370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:35.279894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:35.281227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a1ce3dfc-0ddf-4b97-94e2-abb516278a8c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.185413 Sep 17 04:37:35.281856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 440/873] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:35 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:35.929856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:35.931101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:35.938832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:35.939654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:35.940125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:35.941128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:35.941990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:35.942653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:36.091760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:36.102476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b5352269-87c1-4e93-b16d-3f5890e42e18 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.176410 Sep 17 04:37:36.103475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 434/874] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:35 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:36.307693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:36.308302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:36.315724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:36.316286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:36.316719 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:36.317515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:36.317972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:36.319002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:36.453931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:36.462584 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:36.464318 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:36.466120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-43cfebcc-94e0-4993-9fb9-86303c5c76f5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.162446 Sep 17 04:37:36.466763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 441/875] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:36 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:37:37.125359 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:37.125819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:37.127798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=4 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:37.133791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:37.134259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:37.134692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:37.135371 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:37.135820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:37.136239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:37.276709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:37.284818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0b3e132d-24d3-4327-a10c-22e0601faca7 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.163380 Sep 17 04:37:37.285646 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 435/876] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:37 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:37.492024 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:37.492696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:37.499466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:37.499965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:37.500398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:37.501105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:37.501591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:37.502014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:37.642711 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:37.651151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:37.652572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:37.653866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0936cd94-f025-44cf-aba4-69f4813c8f32 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.165870 Sep 17 04:37:37.654428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 442/877] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:37 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:37.974204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "871c4641-f974-4cc5-bc3e-0c87446fedc6", "name": "network-vif-plugged", "status": "completed", "tag": "b2b856a4-73a0-498f-8867-d7261d331366"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:37.987010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:37.987540 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:37.987966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:38.000325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] Creating event network-vif-plugged:b2b856a4-73a0-498f-8867-d7261d331366 for instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 on hv-cinder-85799 Sep 17 04:37:38.005451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-020a75f7-d41d-403a-99cf-3c87b6f98619 req-7b09f67c-ea0f-4172-8cca-46166ecd4e3c service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036757 Sep 17 04:37:38.006032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 436/878] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:38.308170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:38.308780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:38.314855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:38.315338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:38.317315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:38.318019 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:38.318480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:38.318914 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:38.337273 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:38.462673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:38.470856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8e4b5ecb-8076-4c45-9557-bac255fe3573 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.166652 Sep 17 04:37:38.471487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 443/879] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:38 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:38.680464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:38.681196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:38.687898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:38.688591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:38.689025 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:38.689739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:38.690186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:38.690606 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:38.834613 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:38.843322 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:38.844758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:38.845942 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-485ca3bb-ac3d-44d0-96c1-1912025dab00 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.170434 Sep 17 04:37:38.846508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 437/880] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:38 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:39.497948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:39.498729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:39.501043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:39.501761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=8 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:39.508157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:39.508636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:39.509050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:39.509761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:39.510221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:39.510639 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:39.646595 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:39.655921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c93c956a-c7ab-4f87-880c-7de7c50699ab tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1368 microversion: 2.1 time: 0.164611 Sep 17 04:37:39.656498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 444/881] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:39 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1368 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:39.872109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:39.872728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:39.879587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:39.880081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:39.880518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:39.881251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:39.881712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:39.882137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:40.013632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:40.021898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:40.023348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:40.024632 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a9bcda92-2877-459b-a7f5-f4379f9f6a8f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1741 microversion: 2.20 time: 0.156533 Sep 17 04:37:40.025443 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 438/882] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:39 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1741 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:40.679000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:40.679775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:40.686888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:40.687380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:40.687845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:40.688613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:40.689225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:40.689650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:40.829744 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:40.837738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-72ac6612-482b-4f3b-8afb-ca766941c809 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1557 microversion: 2.1 time: 0.163175 Sep 17 04:37:40.838238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 445/883] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:40 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1557 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:40.860961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:40.861497 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:40.867869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:40.868515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:40.868954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:40.869702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:40.870159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:40.870580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:40.956520 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-88532689-ff53-4923-ab97-a3c9c99f1c24 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099569 Sep 17 04:37:40.957173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 439/884] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:37:40 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:37:41.051052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:41.051688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:41.058099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:41.058580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:41.059010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:41.059696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:41.060143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:41.060565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:41.194093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:41.202500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:41.203946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:41.205140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b7de153d-8c93-428b-982a-7b3192ad163e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1741 microversion: 2.20 time: 0.158682 Sep 17 04:37:41.205704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 446/885] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:41 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1741 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:41.937727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "871c4641-f974-4cc5-bc3e-0c87446fedc6", "tag": "b2b856a4-73a0-498f-8867-d7261d331366"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:41.947096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:41.947565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:41.947984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:41.959116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] Creating event network-changed:b2b856a4-73a0-498f-8867-d7261d331366 for instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 on hv-cinder-85799 Sep 17 04:37:41.964067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-a5cf2e2b-e921-4202-a441-d5440fd5cf5a req-6f178638-c52d-4ea5-9e1d-3bb03ce77c3d service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030500 Sep 17 04:37:41.964729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 440/886] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:42.089624 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "name": "network-vif-plugged", "status": "completed", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:42.101838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:42.102446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:42.103307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:42.120110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] Creating event network-vif-plugged:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:37:42.130366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-508a77ae-41fa-4253-9961-64e8dcc4436e req-db6d0510-e802-4359-98c3-11f6c83c3ca8 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043923 Sep 17 04:37:42.132601 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 447/887] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:42.234092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:42.235523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:42.243757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:42.244533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:42.245297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:42.246207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:42.246902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:42.247697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:42.276433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:42.516589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:42.525947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:42.527586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:42.528879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-29683f79-26b1-4fca-9175-e5f2e2c9980f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1741 microversion: 2.20 time: 0.300905 Sep 17 04:37:42.529931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 441/888] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:42 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1741 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:43.555768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:43.556450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:43.562733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:43.563214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:43.563641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:43.564315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:43.564760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:43.565214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:43.698048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:43.709901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:43.712152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:43.714097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-42689751-eb1c-49d7-8883-eaec7ff98764 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1741 microversion: 2.20 time: 0.162939 Sep 17 04:37:43.715049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 448/889] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:43 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1741 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:44.741245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:44.741881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:44.748565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:44.749052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:44.749536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:44.750223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:44.750673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:44.751111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:44.891311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:44.899609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:44.901042 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:44.902225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-238186b4-4de8-42f7-84c7-9266dcba9c5c tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1741 microversion: 2.20 time: 0.164985 Sep 17 04:37:44.902794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 442/890] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:44 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1741 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:45.928196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:45.928807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:45.943109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:45.943811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:45.944312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:45.945354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:45.945675 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:45.946099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:46.433570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:46.446268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:46.448126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:46.449586 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c056447b-1ba9-4584-a4d1-d49aa29098ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.525211 Sep 17 04:37:46.450254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 449/891] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:45 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:47.478304 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:47.479889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:47.492217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:47.493377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:47.494060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:47.495169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:47.495925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:47.496776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:47.660940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:47.671760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:47.673523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:47.675010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ecc9bccd-eff6-4d91-8819-2aaa7a239f50 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.201173 Sep 17 04:37:47.675672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 443/892] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:47 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:48.706501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:48.707844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:48.714790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:48.715876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:48.716469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:48.717352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:48.717982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:48.718649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:48.877705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:48.892140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:48.895401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:48.896864 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-94537362-431c-40e6-9d6c-03ba4ff3b27d tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.194434 Sep 17 04:37:48.897672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 450/893] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:48 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:49.931866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:49.933328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:49.944277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:49.945063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:49.945685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:49.946533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:49.948997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:49.949665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:50.125750 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:50.134705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:50.143172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:50.144467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-463d6dd1-8028-40e2-8f93-66ce82e764e1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.217708 Sep 17 04:37:50.145124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 444/894] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:49 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:51.171570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:51.172917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:51.180063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:51.180755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:51.181399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:51.182276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:51.182918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:51.183508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:51.329739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:51.343516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:51.345534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:51.346887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e4dedcd1-f05e-4740-b153-5ff43a15af5a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.180451 Sep 17 04:37:51.347420 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 451/895] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:51 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:51.738083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1875461991", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "networks": [{"uuid": "ad7b6fdb-eeaa-4116-adbf-8eccf6c5ce96"}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:51.796802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] validate_networks() for [('ad7b6fdb-eeaa-4116-adbf-8eccf6c5ce96', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:37:52.193469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:37:52.193922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:37:52.194410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:37:52.194818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:37:52.195413 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:37:52.204855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:52.205371 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:52.205807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:52.206642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:52.207102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:52.207661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:52.226381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Getting quotas for project db2188067a23494ab49f1e6812e501d6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:37:52.231324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Getting quotas for user 7cd65e4d3b2f4976be9176e190d78d1d and project db2188067a23494ab49f1e6812e501d6. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:37:52.243473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:37:52.246411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:37:52.294518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2902f132-ff85-44c2-9f04-5dc71a85b3b1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.560339 Sep 17 04:37:52.301298 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 445/896] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:37:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 568 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:37:52.307797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-366f6ab2-193f-44a6-917e-449084a6f097 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:52.308386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-366f6ab2-193f-44a6-917e-449084a6f097 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:52.373175 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:52.374014 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:52.380619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:52.381121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:52.381724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:52.382576 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:52.383053 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:52.383738 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:52.406112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-366f6ab2-193f-44a6-917e-449084a6f097 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:52.414806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-366f6ab2-193f-44a6-917e-449084a6f097 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1214 microversion: 2.1 time: 0.111424 Sep 17 04:37:52.416175 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 452/897] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:52 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1214 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:37:52.547430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:52.567381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:52.570216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:52.571897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-37c7d372-7656-488d-be58-7dace498ca41 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.203521 Sep 17 04:37:52.577865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 446/898] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:52 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:37:53.440314 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:53.441105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:53.448801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:53.449354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:53.449782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:53.450498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:53.450930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:53.451361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:53.526520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:53.535603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-05e38231-0006-4851-9866-bef0e0ad6580 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1266 microversion: 2.1 time: 0.099641 Sep 17 04:37:53.536514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 453/899] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:53 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1266 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:53.601606 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:53.602533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:53.604752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=4 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:53.635453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:53.636257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:53.636805 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:53.644005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:53.644524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:53.644959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:54.212527 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:54.221564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:54.223267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:54.224634 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-addf3c49-1e34-4a58-b147-f7559943bac4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.628348 Sep 17 04:37:54.225236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 447/900] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:53 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:54.562482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:54.563798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:54.566539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=68,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:37:54.572050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:54.572879 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:54.573625 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:54.574602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:54.575362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:54.576049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:54.663551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:54.678374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9e06ba2b-0617-40bb-8b1d-4c3185a8fefc tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1272 microversion: 2.1 time: 0.120100 Sep 17 04:37:54.679531 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 454/901] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:54 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1272 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:55.252932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:55.253724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:55.263078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:55.263689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:55.264377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:55.265777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:55.266200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:55.267127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:55.513700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:55.522047 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:55.523583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:55.524891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c490ce90-629b-41ac-904c-e48bcbbcd473 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1736 microversion: 2.20 time: 0.278839 Sep 17 04:37:55.525496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 448/902] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:55 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1736 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:55.667672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9f66b43-8579-4da4-a345-4507e5aff93b", "tag": "5fa6e0bb-ef83-4b8a-81c6-789cc882210f"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:37:55.679437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:55.680002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:55.680686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:55.692961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] Creating event network-changed:5fa6e0bb-ef83-4b8a-81c6-789cc882210f for instance b9f66b43-8579-4da4-a345-4507e5aff93b on hv-cinder-85799 Sep 17 04:37:55.697946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-05e95241-fdbb-4a92-b0a8-7fb8b08befa3 req-92fd1f74-a2c7-4796-81f8-4bc73fdf1623 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033999 Sep 17 04:37:55.698736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 455/903] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:37:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:37:55.702978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:55.703597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:55.710777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:55.711401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:55.711846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:55.712548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:55.713006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:55.713463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:55.849004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:55.856929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-53c9499c-db13-48eb-b72f-d41ddc075b27 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.158396 Sep 17 04:37:55.857529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 449/904] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:55 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:37:56.550870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:56.551579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:56.558309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:56.558792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:56.559216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:56.559898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:56.560343 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:56.560756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:56.710810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:56.719863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:56.721337 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:56.722501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6dccdee9-0422-4fdd-a696-9245352bca82 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1718 microversion: 2.20 time: 0.176016 Sep 17 04:37:56.723063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 456/905] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:56 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1718 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:56.884117 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:56.885056 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:56.893932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:56.894429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:56.894862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:56.895557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:56.896015 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:56.896442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:57.038971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:57.048508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-57a83d6f-4dcf-49b6-b158-3246d0e21db8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.171121 Sep 17 04:37:57.049059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 450/906] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:56 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:57.760168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:57.761185 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:57.769807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:57.770284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:57.770709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:57.771388 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:57.771832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:57.772248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:57.911500 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:57.924340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:57.926547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:57.928259 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-68a556e0-775e-4908-9436-25ee63495d78 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1718 microversion: 2.20 time: 0.174621 Sep 17 04:37:57.928991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 457/907] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:57 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1718 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:58.075272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:58.076258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:58.087039 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:58.087835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:58.088552 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:58.089725 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:58.090838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:58.091575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:58.225592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:58.235111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d76cb59e-434f-406a-aaa0-c8a1b32a24ab tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.166435 Sep 17 04:37:58.235754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 451/908] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:58 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:37:58.960304 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:58.961180 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:58.971192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:58.971678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:58.972120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:58.972827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:58.973325 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:58.973756 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:59.105003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:59.113662 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:59.115110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:59.116268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bc0887ee-4ead-4ffe-b44e-9d387ba3214f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1718 microversion: 2.20 time: 0.162744 Sep 17 04:37:59.116807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 458/909] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:37:58 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1718 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:37:59.262195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:37:59.263048 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:37:59.270914 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:59.271409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:59.271849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:59.272550 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:37:59.273002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:37:59.273473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:37:59.406812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:37:59.414601 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6be886c2-2b7e-48ad-b51f-e8326ec93742 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.159170 Sep 17 04:37:59.415160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 452/910] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:37:59 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:00.059281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "name": "network-vif-unplugged", "status": "completed", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:00.069806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:00.070284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:00.070701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:00.082575 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] Creating event network-vif-unplugged:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:38:00.088000 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-0e204f7c-f182-4a0b-93f9-510b8bd23b56 req-861d738e-9d17-42e6-aaae-bc002fce2ed8 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032353 Sep 17 04:38:00.088514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 459/911] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:00 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:00.142555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:00.143289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:00.149854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:00.150342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:00.150761 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:00.151457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:00.151983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:00.152429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:00.362995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:00.371876 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:00.373361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:00.374792 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f354b239-02dc-4391-8c3a-6e8505ed80d1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1718 microversion: 2.20 time: 0.236854 Sep 17 04:38:00.375345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 453/912] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:00 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1718 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:00.440051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:00.441184 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:00.450485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:00.451246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:00.451857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:00.452923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:00.453647 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:00.454297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:00.593884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:00.602948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2aa1d42-e56c-4b23-86c0-4d6ff85e18e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.169183 Sep 17 04:38:00.603875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 460/913] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:00 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:01.405518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:01.406560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:01.417534 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.418344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.419071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.420210 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.420968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.421815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.551240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:01.559682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:01.561109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:01.562312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-da7f1ee6-52c0-447b-93cf-5a13580714e3 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1661 microversion: 2.20 time: 0.163597 Sep 17 04:38:01.562891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 454/914] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:01 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1661 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:01.592315 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a13496e2-f6e1-434c-8d49-87594b442a21"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:01.594745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:01.601642 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.602128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.602561 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.603250 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.603697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.604121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.627323 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:01.628037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:01.635181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.635394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.635850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.636547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:01.637005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:01.637654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:01.769420 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:01.777973 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9f4d97ec-9131-4734-90c5-e2dc8fc560ac tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.154952 Sep 17 04:38:01.778533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 455/915] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:01 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:01.802425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:01.894444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f8856d83-3433-49ae-b4ac-a95d87d6296e tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.306496 Sep 17 04:38:01.895080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 461/916] 10.222.0.60 () {62 vars in 1364 bytes} [Sat Sep 17 04:38:01 2022] POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments => generated 188 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 17 04:38:02.028020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:02.028522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:02.034642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:02.035131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:02.035569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:02.036264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:02.036706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:02.037165 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:02.216247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:02.238393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:02.803262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:02.803865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:02.805737 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:02.812699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:02.813260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:02.813691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:02.814371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:02.814823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:02.815239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:02.947390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:02.955688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a9b1d90a-a094-4c51-a81e-afd45abe807a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.156386 Sep 17 04:38:02.956258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 462/917] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:02 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:03.407848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:03.419742 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7eb79ea3-9906-437a-b1a8-a0f635c500d4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments/a13496e2-f6e1-434c-8d49-87594b442a21" status: 202 len: 0 microversion: 2.20 time: 1.395726 Sep 17 04:38:03.420447 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 456/918] 10.222.0.60 () {60 vars in 1457 bytes} [Sat Sep 17 04:38:02 2022] DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments/a13496e2-f6e1-434c-8d49-87594b442a21 => generated 0 bytes in 1397 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:38:03.490088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:03.490716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:03.497066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.497588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.498028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:03.498717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.499167 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.499591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:03.529011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120626) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 17 04:38:03.598911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-023a0e74-23cf-4b4c-bf6c-87d50223a61f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/action" status: 202 len: 0 microversion: 2.20 time: 0.112963 Sep 17 04:38:03.599520 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 463/919] 10.222.0.60 () {62 vars in 1319 bytes} [Sat Sep 17 04:38:03 2022] POST /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:38:03.608226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:03.608743 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:03.615979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.616474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.616901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:03.617620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.618079 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.618501 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:03.759932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:03.768156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:03.769631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:03.770797 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-89d6cab0-95f2-46e7-bbd5-343c55ee6ada tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1669 microversion: 2.20 time: 0.167004 Sep 17 04:38:03.771363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 457/920] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:03 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1669 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:03.979296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:03.980012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:03.986084 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.986566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.987003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:03.987694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:03.988129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:03.988558 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:04.120398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:04.128472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-77b6f145-fe99-4014-8d05-4c2140b5f85c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.153721 Sep 17 04:38:04.129030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 464/921] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:03 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:04.796067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:04.796685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:04.803673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:04.804187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:04.804622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:04.805351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:04.805855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:04.806281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:04.942849 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:04.951024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:04.952470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:04.953660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ffe8683d-0f3e-4779-8b36-f95980c0b593 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1671 microversion: 2.20 time: 0.161650 Sep 17 04:38:04.954244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 458/922] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:04 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1671 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:05.150618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:05.151154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:05.157549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:05.158038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:05.158469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:05.159218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:05.159659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:05.160139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:05.295212 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:05.302984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-78e14dea-f3f7-4b76-af08-d6bd6df86420 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.156569 Sep 17 04:38:05.303562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 465/923] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:05 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:38:05.424794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9f66b43-8579-4da4-a345-4507e5aff93b", "name": "network-vif-plugged", "status": "completed", "tag": "5fa6e0bb-ef83-4b8a-81c6-789cc882210f"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:05.434936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:05.435447 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:05.435834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:05.449297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] Creating event network-vif-plugged:5fa6e0bb-ef83-4b8a-81c6-789cc882210f for instance b9f66b43-8579-4da4-a345-4507e5aff93b on hv-cinder-85799 Sep 17 04:38:05.454361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-5f20a6ce-eb6c-45ac-9865-6f6510d85153 req-0b5ae712-cfe2-4e6e-9a48-18ff2dd25886 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033274 Sep 17 04:38:05.455170 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 459/924] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:05.982110 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:05.982812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:05.989075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:05.989618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:05.990105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:05.990786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:05.991228 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:05.991671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:06.127620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:06.135873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:06.137358 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:06.138536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aeebb175-1e74-46d9-ab6b-82a37a8c3777 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.160441 Sep 17 04:38:06.139099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 466/925] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:05 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:06.329335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:06.330431 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:06.339167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:06.339701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:06.340063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:06.341526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:06.341984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:06.342418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:06.479899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:06.487593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d76022f3-43d8-41ea-b3de-678e8a907a9a tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1314 microversion: 2.1 time: 0.163462 Sep 17 04:38:06.488161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 460/926] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:06 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:07.164311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:07.164927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:07.171929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:07.172428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:07.172861 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:07.173673 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:07.174135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:07.174560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:07.305565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:07.314198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:07.315647 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:07.316812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f16fb68e-470a-4db3-be4d-3d7883f040cf tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.156590 Sep 17 04:38:07.317478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 467/927] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:07 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:07.510903 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:07.512207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:07.519334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:07.519865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:07.520300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:07.525306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:07.525993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:07.526631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:07.665731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:07.673685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-29b8997b-4c6e-418f-9779-ec6214f81d49 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1499 microversion: 2.1 time: 0.167001 Sep 17 04:38:07.674258 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 461/928] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:07 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1499 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:08.344559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:08.345959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:08.353233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:08.354074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:08.354740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:08.355678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:08.356631 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:08.357358 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:08.512240 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:08.521119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:08.522636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:08.523913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b6636e02-d0b1-4242-9c7b-1cf33e720f69 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.183789 Sep 17 04:38:08.537634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 468/929] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:08 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:09.349763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:09.350454 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:09.357776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.358361 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.358864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.359722 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.360259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.360747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.391306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:38:09.479179 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0d43fc0c-dc7c-4377-ba8b-3437dd2b3449 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "DELETE /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 204 len: 0 microversion: 2.1 time: 0.133015 Sep 17 04:38:09.479770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 462/930] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:38:09 2022] DELETE /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:38:09.488197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:09.488801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:09.495567 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.496258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.496840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.497740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.498337 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.498906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.549995 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:09.550769 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:09.556878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.557484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.557989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.558756 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:09.559273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:09.559757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:09.634607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:09.642550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2f011ba5-1c77-46ce-9953-f7ea7fc61d13 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.158342 Sep 17 04:38:09.643766 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 469/931] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:09 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:09.703411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:09.711745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:09.713262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:09.714570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-50bc8d67-9cae-4b72-9576-da207071c43a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.169192 Sep 17 04:38:09.715498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 463/932] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:09 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:38:10.676502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:10.677838 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:10.686373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:10.686976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:10.687488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:10.688206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:10.688813 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:10.689513 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:10.741934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:10.742627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:10.750200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:10.750708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:10.751240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:10.751959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:10.752794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:10.753024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:10.843376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:10.852082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9c1aab13-bc87-448b-9ee6-b7d9c3ed4492 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.187608 Sep 17 04:38:10.853154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 470/933] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:10 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:10.915928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:10.929394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:10.930847 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:10.932018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1e16d698-59cd-4354-9bb5-fd2e5d3f90b2 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.195734 Sep 17 04:38:10.941131 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 464/934] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:10 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:11.862514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1153090553", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-623666681", "security_groups": [{"name": "tempest-secgroup-smoke-2116056742"}], "block_device_mapping_v2": [{"uuid": "25fe6be8-70fa-4a4d-9013-cbb1608377dd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e322731d-9c63-4618-8051-e8dea6bb2db6"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:11.875748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:11.877101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:11.886557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:11.886968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:11.887557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:11.888315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:11.888772 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:11.889224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:12.001695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] validate_networks() for [('e322731d-9c63-4618-8051-e8dea6bb2db6', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:38:12.033912 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:12.042139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9a44ea08-a690-44e2-b217-1e3b16d38cc1 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.170205 Sep 17 04:38:12.057108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 465/935] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:11 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:38:12.057108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:12.057108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:12.057108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:12.057692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:12.057692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:12.057692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:12.057692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:12.058034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:12.200549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:12.211318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:12.212782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:12.214019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d307f2cd-008f-4498-867e-78a3ac43b18b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.169940 Sep 17 04:38:12.229436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 466/936] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:12 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:12.428224 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:38:12.428732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:38:12.429290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:38:12.429857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:38:12.430342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:38:12.439621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:12.440699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:12.441463 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:12.442419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:12.443096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:12.444010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:12.462762 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:38:12.467489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Getting quotas for user ff241cbd3f7c47328ac5ef2ed4c66e18 and project 752e65efe679458fab6b8dbc3a9de82f. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:38:12.528427 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:38:12.530659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='25fe6be8-70fa-4a4d-9013-cbb1608377dd',volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:38:12.664832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-82e9a312-9e18-423e-980f-d6fc6b0ed20e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.806328 Sep 17 04:38:12.665653 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 471/937] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:38:11 2022] POST /compute/v2.1/servers => generated 406 bytes in 808 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:38:12.677289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9e8d7e81-30e7-46ed-bb15-b5a65e50f286 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:12.677824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9e8d7e81-30e7-46ed-bb15-b5a65e50f286 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:12.753389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9e8d7e81-30e7-46ed-bb15-b5a65e50f286 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:12.765732 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9e8d7e81-30e7-46ed-bb15-b5a65e50f286 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1094 microversion: 2.1 time: 0.092785 Sep 17 04:38:12.766330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 467/938] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:12 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1094 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:13.070612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:13.071206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:13.078099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.078964 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.079686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.080729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.081379 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.082056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.226473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:13.235943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-34a4060b-e98a-44e4-87d0-f71fe3b85b18 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.171103 Sep 17 04:38:13.236690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 472/939] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:13 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:13.239945 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:13.240668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:13.247874 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.248364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.248802 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.249523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.250267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.250846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.385412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:13.394517 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:13.396472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:13.398166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d011319f-2cef-45aa-b5da-d18729522bc1 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.162607 Sep 17 04:38:13.398726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 468/940] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:13 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:13.792879 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:13.793542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:13.800250 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.800808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.801325 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.802076 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:13.802555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:13.802979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:13.867488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:13.876041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a86beab3-bc49-4904-813b-1e34a6d1152b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.087219 Sep 17 04:38:13.877038 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 473/941] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:13 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:14.258406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:14.259326 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:14.266784 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.267619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.268299 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.269440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.270253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.270889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.410515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:14.419007 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-236f8fd0-a23d-4e45-9875-5a0268f736d0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.164532 Sep 17 04:38:14.419867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 469/942] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:14 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:14.423486 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:14.424096 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:14.430629 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.431106 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.431524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.432200 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.432674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.433115 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.577754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:14.586090 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:14.587536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:14.588723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-060b8aa5-df3b-446b-9968-0a89cc809589 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.169415 Sep 17 04:38:14.589338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 474/943] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:14 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:38:14.899591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:14.900211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:14.906665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.907157 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.907595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.908300 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:14.908757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:14.909251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:14.972070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:14.980564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3ba8c511-a60f-4980-8da0-5ae766ce713d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.085018 Sep 17 04:38:14.981408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 470/944] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:14 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:15.443120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:15.444651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:15.453060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:15.454099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:15.454863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:15.455885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:15.456566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:15.457333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:15.614544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:15.615052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:15.615174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:15.622218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:15.622666 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:15.623104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:15.623287 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e7041e68-5a1d-4942-8d04-c9b9526b6c9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.184316 Sep 17 04:38:15.623807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:15.623948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 475/945] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:15 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:15.624551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:15.625107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:15.763461 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:15.772059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:15.773558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:15.774733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-303e54e1-a52f-4c41-877c-cbfad984b1c9 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.165009 Sep 17 04:38:15.775379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 471/946] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:15 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:16.008074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:16.009146 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:16.019733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.020702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.021614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.022890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.023796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.024502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.088441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:16.092451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:16.097961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-743b8644-9261-4731-81a2-d31bf546e8d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.096876 Sep 17 04:38:16.098552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 476/947] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:15 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:16.647512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:16.648095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:16.654786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.655268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.656163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.656920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.657130 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.657638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.799932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:16.800559 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:16.808388 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.808858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.809344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.810032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:16.810482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:16.810900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:16.813588 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:16.821559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d64d1cc5-4617-4961-8441-08972f4da969 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.177933 Sep 17 04:38:16.822167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 472/948] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:16 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:16.959221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:16.969414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:16.971028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:16.972322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bfa73e72-2986-4170-9fcd-257a381e4194 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.176317 Sep 17 04:38:16.972857 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 477/949] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:16 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:38:17.121411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:17.122031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:17.130500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:17.131317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:17.131976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:17.132752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:17.133317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:17.133831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:17.136950 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:17.200671 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:17.208610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3bfaa78d-c723-4baf-9f6d-6192b645f2a2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.091328 Sep 17 04:38:17.209160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 473/950] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:17 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:17.847742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:17.848376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:17.855548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:17.856032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:17.856507 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:17.857246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:17.857709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:17.858138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:17.992060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:17.997356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:17.998069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:18.000268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f73b096a-8ef8-4edf-a90b-5d0e1ca6de9e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.156492 Sep 17 04:38:18.000847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 478/951] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:17 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:18.004887 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:18.005425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:18.005901 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:18.006607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:18.007051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:18.007478 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:18.152453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:18.161758 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:18.163193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:18.164357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-41cfd3a8-e37f-4158-b238-54a215ed91d5 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.170986 Sep 17 04:38:18.165145 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 474/952] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:17 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:18.232381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:18.232957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:18.239649 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:18.240588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:18.240588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:18.241319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:18.241742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:18.242164 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:18.313031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:18.324457 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-6bf1d2e2-fb0e-4131-8688-3743bc1c50a9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.095969 Sep 17 04:38:18.325255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 479/953] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:18 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:19.027774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:19.028747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:19.039675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.040551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.041302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.042428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.043174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.043869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.179999 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:19.188329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-66f3b8d7-316c-49a7-b277-0e840b105490 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.167202 Sep 17 04:38:19.188755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:19.189328 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 475/954] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:19 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:19.189437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:19.196129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.196615 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.197057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.197807 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.198393 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.198859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.347233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:19.347970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:19.351105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:19.354744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.355318 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.355829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.356578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:19.357185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:19.357690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:19.359558 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:19.360977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:19.362119 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b38a59b1-6c4e-47f1-a3fa-c6cdad642691 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.177317 Sep 17 04:38:19.362699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 480/955] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:19 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:19.422075 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:19.430314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bdbae4a8-f4ca-4b1d-9b98-fd1fb66c7ea9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.087075 Sep 17 04:38:19.430892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 476/956] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:19 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:20.216591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:20.217701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:20.229236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.229814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.230460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.231281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.231729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.232193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.363896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:20.371728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9d339e4c-b853-40a0-827e-b6da418e1e72 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.162351 Sep 17 04:38:20.372255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 481/957] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:20 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:20.387794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:20.388445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:20.395346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.395828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.396269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.396938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.397422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.397845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.453381 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:20.454097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:20.460437 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.460927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.461460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.462159 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:20.462609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:20.463037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:20.530055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:20.530581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:20.538353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-17a6dee2-2024-4188-b515-bdd85f3eb3c5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.088981 Sep 17 04:38:20.538606 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:20.538931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 482/958] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:20 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:20.540098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:20.541423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-30f91e8e-4e90-47cd-975e-fe617d056d0b tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.158030 Sep 17 04:38:20.541978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 477/959] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:20 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:21.399196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:21.400214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:21.408706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.409363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.409906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.410762 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.411327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.411859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.562218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:21.562867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:21.569811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.570321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.570781 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.571471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.571947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.572364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.600290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:21.608749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-26ce692a-9a83-4d7c-ae8f-e3bcac7ba078 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.216479 Sep 17 04:38:21.609369 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 483/960] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:21 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:21.614123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:21.614746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:21.620776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.621431 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.621869 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.622572 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:21.623030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:21.623432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:21.633105 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:21.641358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-547cdac6-d312-45a4-b737-d3081585c175 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.083403 Sep 17 04:38:21.642154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 478/961] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:21 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:21.762432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:21.770893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:21.772313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:21.773492 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-67fae854-c327-4dc5-9a97-360a68a09f85 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.163206 Sep 17 04:38:21.774050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 484/962] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:21 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:22.635486 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:22.636074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:22.643111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.643595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.644036 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.644751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.645266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.645692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.672451 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:22.673191 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:22.675680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:22.676445 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:22.682904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.683773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.684218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.684912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.685402 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.685830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.750802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:22.757846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:22.760410 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b56515b5-c526-4d9b-b5f1-c87cb7040053 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.091986 Sep 17 04:38:22.761865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 485/963] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:22 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:22.798805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:22.799414 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:22.805504 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.805993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.806542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.807223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:22.807663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:22.808445 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:22.829824 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:22.838391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b2b85234-1425-48fc-9d9c-fe32bd913344 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.208958 Sep 17 04:38:22.838960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 479/964] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:22 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:22.939153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:22.947571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:22.949007 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:22.950202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c9a3b2cf-4d37-43c7-9044-c1ec2bd2a629 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.155448 Sep 17 04:38:22.950762 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 486/965] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:22 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:23.788898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:23.789970 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:23.792911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:23.802055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.802863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.803593 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:23.804732 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.805536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.806250 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:23.862073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:23.862795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:23.870761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.871312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.871437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:23.871948 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:23.873113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.873847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.874634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:23.881710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-06cf7430-ff6e-4067-b3d5-4e86edc5b776 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1144 microversion: 2.1 time: 0.099353 Sep 17 04:38:23.882677 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 480/966] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:23 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1144 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:23.975899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:23.976507 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:23.982944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.983440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.983877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:23.984575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:23.985031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:23.985487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:24.018739 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:24.027087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7e7ff941-7681-4dea-8e26-b4661e538ff8 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.168965 Sep 17 04:38:24.028006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 487/967] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:23 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:24.132528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:24.141115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:24.142543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:24.143712 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e85e866e-ca00-4f5b-a13e-0ce238b933a4 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.171890 Sep 17 04:38:24.144240 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 481/968] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:23 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:24.910099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:24.911137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:24.914764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:24.921048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:24.921594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:24.922020 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:24.922705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:24.923156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:24.923641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:24.993680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:25.002235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-47c09ce7-96a8-4701-8711-1cf61615e65c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1136 microversion: 2.1 time: 0.098845 Sep 17 04:38:25.002808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 488/969] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:24 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1136 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:25.053873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:25.054490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:25.062379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:25.062869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:25.063307 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:25.064002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:25.064463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:25.064892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:25.171480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:25.172097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:25.178698 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:25.179125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:25.179571 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:25.180256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:25.180772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:25.181250 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:25.196461 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:25.204634 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0ddaf2ff-9dfc-4ff4-89a2-bfad2bddeae9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.157035 Sep 17 04:38:25.205277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 482/970] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:25 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:25.311843 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:25.320603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:25.322083 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:25.323258 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a4321eae-5fc0-4a96-9b9c-3bddac160c80 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.155790 Sep 17 04:38:25.323821 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 489/971] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:25 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:26.028491 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:26.029128 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:26.035476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.035966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.036402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.037115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.037590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.037984 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.100005 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:26.108143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d8233f7a-0549-4532-ac51-aad1535b5a77 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1192 microversion: 2.1 time: 0.084783 Sep 17 04:38:26.108681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 483/972] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:26 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1192 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:26.230980 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:26.231597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:26.238453 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.238917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.239384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.240112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.240609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.241040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.349962 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:26.351317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:26.357596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.358214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.358706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.359475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.360026 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.360482 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.442582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:26.452286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-315168de-a59a-4119-b6a2-ccbfb5e7790d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1505 microversion: 2.1 time: 0.226697 Sep 17 04:38:26.453516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 490/973] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:26 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1505 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:26.575498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:26.584256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:26.585737 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:26.587063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-16e15d79-41d1-468b-86e5-4da34efda641 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.242045 Sep 17 04:38:26.588302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 484/974] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:26 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:38:26.809206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9f66b43-8579-4da4-a345-4507e5aff93b", "name": "network-vif-deleted", "tag": "5fa6e0bb-ef83-4b8a-81c6-789cc882210f"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:26.820826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:26.821430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:26.821935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:26.844805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] Creating event network-vif-deleted:5fa6e0bb-ef83-4b8a-81c6-789cc882210f for instance b9f66b43-8579-4da4-a345-4507e5aff93b on hv-cinder-85799 Sep 17 04:38:26.850524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-ae0567d6-65b7-44ca-bc2f-b882f2cde689 req-f568b8b9-313c-413b-bb58-0b9646e58b56 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045242 Sep 17 04:38:26.851138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 491/975] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:27.131955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:27.132544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:27.140940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.141748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.142402 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.143140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.143572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.144000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.290747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:27.302253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-32f813b1-85c9-4849-844f-cd037241b76e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1278 microversion: 2.1 time: 0.175117 Sep 17 04:38:27.302856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 485/976] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:27 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1278 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:27.463340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "name": "network-vif-plugged", "status": "completed", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:27.478143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:27.479755 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:27.487868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.488330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.488751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.488876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.489546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.490526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.491469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.492268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.492963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.513037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] Creating event network-vif-plugged:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:38:27.524166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-4b5aea91-ac64-43f5-8385-038c5bdaa31d req-63bcd63c-fcc6-4278-9d4f-23a8b0379a26 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063832 Sep 17 04:38:27.524993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 492/977] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:27.537074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.538016 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.538458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.539098 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lazy-loading 'fault' on Instance uuid b9f66b43-8579-4da4-a345-4507e5aff93b {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:38:27.582223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:27.590424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6a5b7067-4093-45cd-b5a4-d6cd97424f58 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 200 len: 1304 microversion: 2.1 time: 0.118075 Sep 17 04:38:27.591008 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 486/978] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:27 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 1304 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:27.617992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:27.618892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:27.625669 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.626242 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.626751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.627530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:27.628057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:27.628603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:27.865705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:27.874771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:27.876216 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:27.877429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-149e7b18-aa81-4ea9-b47b-f91ece9765b6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.265727 Sep 17 04:38:27.877993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 493/979] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:27 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:28.234747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bc4b19eb-58f5-49b5-8a87-74126c73513f", "tag": "db2efe3b-92b0-4a7f-9c0c-416d334db906"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:28.244833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.245417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.245856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.256808 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] Creating event network-changed:db2efe3b-92b0-4a7f-9c0c-416d334db906 for instance bc4b19eb-58f5-49b5-8a87-74126c73513f on hv-cinder-85799 Sep 17 04:38:28.261767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-37e22bc7-b01a-4d52-90ff-c0246b46c9dd req-c321b756-40f8-449a-aef2-3fc8939508f7 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030775 Sep 17 04:38:28.262622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 487/980] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:28.326428 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:28.327044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:28.333274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.333764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.334200 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.334889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.335345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.335765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.488892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:28.493477 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=100 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:28.498886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-08d707c9-2d4b-4580-83c3-bb54a01449d9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.176340 Sep 17 04:38:28.499489 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 494/981] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:28 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:28.617976 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:28.619039 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: b9f66b43-8579-4da4-a345-4507e5aff93b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:28.628516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.629345 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.630031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.631037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.631727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.632364 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.648660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] HTTP exception thrown: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Request method failure captured: Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b HTTP/1.1 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU8d0BB8sZXxUnmDaMRxdOW8KgMv_m0RYlGlovBPJH7AtLKhIqlcxi_Y5hnDcjLVoZcgHVgV1avQJ3C4JNgpU_ON0ojAwNGPZ5fPsKlwYQjoGrZuVLekd3sbfLPjQ15OClNPLu4lDO6dTI4mLvab4kT95uuQI_zrlQ_h30jzRUY8dyY Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: db2188067a23494ab49f1e6812e501d6 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesActionsTest-965492678 Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:38:28.650344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_db2188067a23494ab49f1e6812e501d6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesActionsTest-965492678 Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: db2188067a23494ab49f1e6812e501d6 Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesActionsTest-965492678 Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesActionsTest-965492678-project Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 7cd65e4d3b2f4976be9176e190d78d1d Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesActionsTest-965492678-project Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. Sep 17 04:38:28.652040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'b9f66b43-8579-4da4-a345-4507e5aff93b'} Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:38:28.656286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. Sep 17 04:38:28.657891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:38:28.659430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Returning 404 to user: Instance b9f66b43-8579-4da4-a345-4507e5aff93b could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:38:28.659430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e37f9dc2-f2e5-4e94-8245-688c88ddfe3d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b" status: 404 len: 111 microversion: 2.1 time: 0.040035 Sep 17 04:38:28.659430 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 488/982] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:28 2022] GET /compute/v2.1/servers/b9f66b43-8579-4da4-a345-4507e5aff93b => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:38:28.709830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1158687362", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "networks": [{"uuid": "ad7b6fdb-eeaa-4116-adbf-8eccf6c5ce96"}]}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:28.767424 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] validate_networks() for [('ad7b6fdb-eeaa-4116-adbf-8eccf6c5ce96', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:38:28.902766 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:28.903379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:28.909626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.910112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.910545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:28.911212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:28.911697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:28.912121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:29.042509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:29.050992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:29.052429 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:29.053626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2296907e-ed94-477a-a6fe-12901f5ae4ae tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.154986 Sep 17 04:38:29.054178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 489/983] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:28 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:38:29.244059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:38:29.244518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:38:29.244973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:38:29.245394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:38:29.245829 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:38:29.255487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:29.255955 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:29.256384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:29.257122 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:29.257593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:29.258076 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:29.276274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Getting quotas for project db2188067a23494ab49f1e6812e501d6. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:38:29.280695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Getting quotas for user 7cd65e4d3b2f4976be9176e190d78d1d and project db2188067a23494ab49f1e6812e501d6. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:38:29.292366 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:38:29.294763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:38:29.345505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-abe33464-9cde-481b-bc4e-871feb7d087d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.639527 Sep 17 04:38:29.346155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 495/984] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:38:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 641 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:38:29.357623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c6a8606c-5297-4a61-b915-f8e3bb2ba6da tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:29.358217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c6a8606c-5297-4a61-b915-f8e3bb2ba6da tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:29.415108 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c6a8606c-5297-4a61-b915-f8e3bb2ba6da tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:29.423002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c6a8606c-5297-4a61-b915-f8e3bb2ba6da tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1214 microversion: 2.1 time: 0.069294 Sep 17 04:38:29.424135 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 490/985] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:29 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1214 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:38:29.527881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:29.528491 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:29.535643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:29.536153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:29.536588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:29.537305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:29.537754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:29.538172 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:29.686752 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:29.694779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-04aff01f-0906-4856-bb93-d4bba821e34d tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.171386 Sep 17 04:38:29.695352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 496/986] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:29 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:30.080115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:30.080716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:30.089281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.090224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.090728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.091557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.092180 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.092608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.447849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:30.448880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:30.452020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:30.454795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:30.462099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:30.463516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.464044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:30.465188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.465630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.466094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-95e636f2-21ab-435b-aaa2-078495f662ed tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1727 microversion: 2.20 time: 0.390600 Sep 17 04:38:30.466863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.469925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.470389 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.477678 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 491/987] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:30 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1727 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:30.545876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:30.553771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bda217bb-0bd7-4d35-911e-8f7ece1aa995 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1266 microversion: 2.1 time: 0.111102 Sep 17 04:38:30.554436 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 497/988] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:30 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1266 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:30.721109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:30.721924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:30.730274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.730831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.731266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.731964 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:30.732421 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:30.732851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:30.871357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:30.881566 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4dfaa18a-91f7-4b93-9ddd-502a7d368cc2 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.166445 Sep 17 04:38:30.882944 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 492/989] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:30 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:31.494129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:31.494859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:31.502536 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.503145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.503693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:31.504550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.505143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.505721 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:31.577121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:31.577715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:31.584778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.585301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.585734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:31.586432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.586883 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.587305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:31.643380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:31.652303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:31.653867 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:31.655118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-521062ce-1a7d-47ae-98dd-689aa56e2bb6 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1714 microversion: 2.20 time: 0.166141 Sep 17 04:38:31.655764 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 498/990] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:38:31 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1714 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:38:31.659057 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:31.667200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d71d9bf8-1c65-4398-aa88-9b176264fad4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1284 microversion: 2.1 time: 0.094050 Sep 17 04:38:31.667869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 493/991] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:31 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:31.905927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:31.906542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:31.913227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.913732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.914828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:31.915555 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:31.916014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:31.916442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:32.096161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:32.103972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fe4b8000-f21a-4d33-8b8d-4d737c030d8e tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.202113 Sep 17 04:38:32.104661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 499/992] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:31 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:32.690770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:32.691409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:32.698776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:32.699192 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:32.699625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:32.700302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:32.700762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:32.701216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:32.829308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:32.837485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-add7b10b-b98d-4949-a1e3-72586591d709 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.150845 Sep 17 04:38:32.838277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 494/993] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:32 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:33.127767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:33.128340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:33.135695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:33.136262 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:33.136697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:33.137425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:33.137875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:33.138297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:33.220988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8cf771b6-8252-4ad3-8573-042feb9b906b", "tag": "e1498365-30de-4531-b9f2-2ba9486cfb08"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:33.230389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:33.230890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:33.231317 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:33.242262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] Creating event network-changed:e1498365-30de-4531-b9f2-2ba9486cfb08 for instance 8cf771b6-8252-4ad3-8573-042feb9b906b on hv-cinder-85799 Sep 17 04:38:33.247391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-32eadbff-40d0-41f6-9761-f1e45f49ae03 req-c0e47e14-eaed-4b32-96fb-8bb1acaaa0a5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031596 Sep 17 04:38:33.248077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 495/994] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:38:33.373316 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:33.381550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-37b93861-80f1-4a99-9ef0-427d8fea67e9 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.257712 Sep 17 04:38:33.382133 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 500/995] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:33 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:33.866800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:33.867960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:33.878392 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:33.879206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:33.879931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:33.881067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:33.881978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:33.882689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:34.017223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:34.025289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a359be3a-8805-41cb-9a65-c926f885bcb0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.166448 Sep 17 04:38:34.025859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 496/996] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:33 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:34.408292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:34.409346 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:34.418618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:34.419106 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:34.419539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:34.420261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:34.420717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:34.421151 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:34.563220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:34.571284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2db5f115-3d3f-415e-bad2-a496a1165af1 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.169876 Sep 17 04:38:34.571855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 501/997] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:34 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:35.051029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:35.052004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:35.060942 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:35.061460 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:35.061898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:35.062595 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:35.063050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:35.063485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:35.207352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:35.215813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-807b3be0-502a-4f6a-be63-efdbdb526118 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.171275 Sep 17 04:38:35.216385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 497/998] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:35 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:35.599579 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:35.600612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:35.611715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:35.612515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:35.613283 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:35.614425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:35.615188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:35.615890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:35.755721 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:35.763669 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d67a56f4-21e6-4195-9606-5971fc585ae4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.170814 Sep 17 04:38:35.764280 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 502/999] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:35 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:36.239124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:36.239708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:36.246832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:36.247411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:36.247848 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:36.248604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:36.249062 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:36.249519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:36.666374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:36.693877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1ced5ae8-f36d-4451-8550-f338629c700d tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.458731 Sep 17 04:38:36.696389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 498/1000] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:36 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:36.787249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:36.788008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:36.794884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:36.795341 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:36.795776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:36.796473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:36.797036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:36.797435 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:36.965364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:36.973806 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d753f7ea-4ed0-4303-95cc-f129feee36f0 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.190691 Sep 17 04:38:36.974396 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 503/1001] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:36 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:37.721671 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:37.722522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:37.732253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:37.733052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:37.733820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:37.734979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:37.735739 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:37.736453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:37.874564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:37.886334 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-decf9b34-c61f-4bb1-9951-d0836520c7aa tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.171324 Sep 17 04:38:37.887041 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 499/1002] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:37 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:38.000925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:38.001805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:38.012902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:38.013745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:38.014466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:38.015594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:38.016340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:38.017036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:38.166398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:38.175230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5e9be7c6-31d1-47bf-a3da-205cc3a40441 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.180994 Sep 17 04:38:38.175792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 504/1003] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:37 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:38.912330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:38.912923 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:38.919985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:38.920518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:38.920943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:38.921757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:38.922161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:38.922625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:39.057001 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:39.065238 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-355cfbe5-1008-465c-a553-6eb182191f7e tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.156850 Sep 17 04:38:39.065801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 500/1004] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:38 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:39.199132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:39.199827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:39.208685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:39.209382 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:39.209963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:39.210852 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:39.211460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:39.212076 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:39.265884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:39.286970 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:39.351848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:39.360406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-64c83ebc-45f9-411b-941d-c762fb349b14 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.165550 Sep 17 04:38:39.360963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 505/1005] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:39 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:39.492282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8cf771b6-8252-4ad3-8573-042feb9b906b", "name": "network-vif-plugged", "status": "completed", "tag": "e1498365-30de-4531-b9f2-2ba9486cfb08"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:39.497427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:39.503916 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:39.504391 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:39.504816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:39.516115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] Creating event network-vif-plugged:e1498365-30de-4531-b9f2-2ba9486cfb08 for instance 8cf771b6-8252-4ad3-8573-042feb9b906b on hv-cinder-85799 Sep 17 04:38:39.521291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-23d4baf3-fb31-4ad0-b389-69c7864d3f67 req-b4e2fcac-e1bd-450a-aa15-b48030392e12 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032726 Sep 17 04:38:39.521890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 501/1006] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:40.091120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:40.091730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:40.098444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:40.098934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:40.099374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:40.100073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:40.100532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:40.100961 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:40.235991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:40.244044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-24be9f40-5e7f-46bf-b8cd-cba274780ac4 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.158921 Sep 17 04:38:40.244614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 506/1007] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:40 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:40.385342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:40.385989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:40.392706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:40.393221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:40.393651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:40.394353 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:40.394779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:40.395226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:40.528122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:40.536409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bfe03442-507c-41cc-8898-033c11405d7f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.156201 Sep 17 04:38:40.536990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 502/1008] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:40 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:41.268716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:41.269470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:41.277963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:41.278527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:41.279045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:41.279890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:41.280418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:41.280917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:41.413681 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:41.423502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-49862001-4686-43b7-b597-15e58b691109 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1314 microversion: 2.1 time: 0.159194 Sep 17 04:38:41.424362 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 507/1009] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:41 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:41.560895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:41.561810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:41.568696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:41.569287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:41.569734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:41.570424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:41.570889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:41.571311 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:41.711761 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:41.720631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2aced80c-537a-4c88-93c5-d3354226c874 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.164644 Sep 17 04:38:41.721227 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 503/1010] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:41 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:42.451059 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:42.452093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:42.463196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:42.463997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:42.465075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:42.466026 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:42.466478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:42.466924 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:42.612074 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:42.620012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-56c0b44c-96a7-4531-b0cd-e556cf34a1e0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1498 microversion: 2.1 time: 0.175727 Sep 17 04:38:42.620550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 508/1011] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:42 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1498 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:42.743863 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:42.744383 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:42.751132 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:42.751661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:42.752167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:42.752936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:42.753436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:42.753856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:42.888301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:42.896188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c44a4282-0e52-447c-a118-5d8adcc764ea tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.156260 Sep 17 04:38:42.896804 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 504/1012] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:42 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:43.228751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:43.229375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:43.235537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.236021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.236454 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.237169 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.237620 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.238044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.262890 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:38:43.354109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d2580eaf-31be-44b1-8122-2afd54e684c9 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 204 len: 0 microversion: 2.1 time: 0.129071 Sep 17 04:38:43.354900 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 509/1013] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:38:43 2022] DELETE /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:38:43.363110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:43.363718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:43.369685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.370163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.370594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.371275 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.371774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.372193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.512760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:43.518778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bc4b19eb-58f5-49b5-8a87-74126c73513f", "name": "network-vif-plugged", "status": "completed", "tag": "db2efe3b-92b0-4a7f-9c0c-416d334db906"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:43.522382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ce320c4d-9f7a-437d-bf75-f6d9484e9358 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1504 microversion: 2.1 time: 0.163148 Sep 17 04:38:43.523264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 505/1014] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:43 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1504 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:43.531493 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.532046 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.532404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.544120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] Creating event network-vif-plugged:db2efe3b-92b0-4a7f-9c0c-416d334db906 for instance bc4b19eb-58f5-49b5-8a87-74126c73513f on hv-cinder-85799 Sep 17 04:38:43.549004 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-bb23cab2-50c1-44e2-8505-51dd2a8129f7 req-549cfde2-8cd1-49a1-9253-850a5d5eb4de service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034255 Sep 17 04:38:43.549643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 510/1015] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:38:43.923398 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:43.924439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:43.934822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.935305 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.935740 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:43.936428 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:43.936880 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:43.937330 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:44.076167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:44.084022 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3584d4b6-1018-445b-9f7b-b17b073babe4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.167248 Sep 17 04:38:44.084570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 506/1016] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:43 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:44.551728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:44.552886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:44.562128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:44.562614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:44.563049 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:44.563748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:44.564222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:44.564623 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:44.568742 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:44.702963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:44.711124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9e8f0137-34e0-4129-aced-c7b877395e95 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1504 microversion: 2.1 time: 0.167523 Sep 17 04:38:44.711695 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 511/1017] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:44 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1504 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:45.107561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:45.108152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:45.115069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:45.115559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:45.115992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:45.116681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:45.117136 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:45.117559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:45.252174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:45.264724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dd363848-3da6-4123-a15a-9589cabc5131 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.160831 Sep 17 04:38:45.265628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 507/1018] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:45 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:45.737028 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:45.737934 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:45.747259 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:45.747941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:45.748546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:45.749714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:45.750351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:45.750945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:45.886013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:45.894182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f3f44abf-87f7-4221-8304-44f2f3a5266c tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1504 microversion: 2.1 time: 0.162761 Sep 17 04:38:45.894752 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 512/1019] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:45 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1504 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:46.291635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:46.292262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:46.299424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:46.299904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:46.300332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:46.301012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:46.301488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:46.301904 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:46.439172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:46.447024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b68fc8e8-0f10-46af-88da-ad164b103df7 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1266 microversion: 2.1 time: 0.161781 Sep 17 04:38:46.447627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 508/1020] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:46 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:46.670607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8cf771b6-8252-4ad3-8573-042feb9b906b", "name": "network-vif-deleted", "tag": "e1498365-30de-4531-b9f2-2ba9486cfb08"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:46.681133 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:46.681827 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:46.682254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:46.693227 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] Creating event network-vif-deleted:e1498365-30de-4531-b9f2-2ba9486cfb08 for instance 8cf771b6-8252-4ad3-8573-042feb9b906b on hv-cinder-85799 Sep 17 04:38:46.698044 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-5c1e3b79-2f82-4ced-a052-9a472820ba37 req-5a58548d-c1ad-4288-9707-c10ccc0dbbd0 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033709 Sep 17 04:38:46.698607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 513/1021] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:38:46.921519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:46.922040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:46.931070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:46.931562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:46.932006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:46.932979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:46.933826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:46.934525 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:46.998438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7cd65e4d3b2f4976be9176e190d78d1d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db2188067a23494ab49f1e6812e501d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:47.000833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=100 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:38:47.009806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1d0d4f3b-bc32-4c58-a03c-fbedd7c777ea tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 200 len: 1462 microversion: 2.1 time: 0.094750 Sep 17 04:38:47.010660 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 509/1022] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:46 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 1462 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:47.472498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:47.474014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:47.481848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:47.482331 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:47.482765 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:47.483458 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:47.483909 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:47.484330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:47.623685 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:47.631862 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ba9d228f-4aa6-4d19-9653-19063f0536f4 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1453 microversion: 2.1 time: 0.164714 Sep 17 04:38:47.632435 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 514/1023] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:47 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1453 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:47.660194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:47.660851 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:47.668752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:47.669370 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:47.669810 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:47.670516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:47.671011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:47.671445 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:47.803164 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:38:47.811404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-efee0966-802a-46e1-a7e3-6ebe02bd3f97 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1453 microversion: 2.1 time: 0.155259 Sep 17 04:38:47.811972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 510/1024] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:47 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1453 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:38:48.034303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:38:48.034949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] [instance: 8cf771b6-8252-4ad3-8573-042feb9b906b] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:38:48.041276 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:48.041767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:48.042203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:48.042895 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:48.043348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:48.043773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:48.058116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] HTTP exception thrown: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Request method failure captured: Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b HTTP/1.1 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU8d0BB8sZXxUnmDaMRxdOW8KgMv_m0RYlGlovBPJH7AtLKhIqlcxi_Y5hnDcjLVoZcgHVgV1avQJ3C4JNgpU_ON0ojAwNGPZ5fPsKlwYQjoGrZuVLekd3sbfLPjQ15OClNPLu4lDO6dTI4mLvab4kT95uuQI_zrlQ_h30jzRUY8dyY Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: db2188067a23494ab49f1e6812e501d6 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesActionsTest-965492678 Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:38:48.059177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_db2188067a23494ab49f1e6812e501d6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/db2188067a23494ab49f1e6812e501d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesActionsTest-965492678 Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: db2188067a23494ab49f1e6812e501d6 Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesActionsTest-965492678 Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesActionsTest-965492678-project Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 7cd65e4d3b2f4976be9176e190d78d1d Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesActionsTest-965492678-project Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. Sep 17 04:38:48.060588 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '8cf771b6-8252-4ad3-8573-042feb9b906b'} Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:38:48.064255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. Sep 17 04:38:48.065731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:38:48.067017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] Returning 404 to user: Instance 8cf771b6-8252-4ad3-8573-042feb9b906b could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:38:48.067017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-40641e4c-aca9-419e-857c-dd68ddae6bc0 tempest-VolumesActionsTest-965492678 tempest-VolumesActionsTest-965492678-project] 10.222.0.60 "GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b" status: 404 len: 111 microversion: 2.1 time: 0.030283 Sep 17 04:38:48.067017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 515/1025] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:38:48 2022] GET /compute/v2.1/servers/8cf771b6-8252-4ad3-8573-042feb9b906b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:38:49.565251 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bc4b19eb-58f5-49b5-8a87-74126c73513f", "tag": "db2efe3b-92b0-4a7f-9c0c-416d334db906"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:38:49.575480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:38:49.575959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:38:49.576384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:38:49.589050 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] Creating event network-changed:db2efe3b-92b0-4a7f-9c0c-416d334db906 for instance bc4b19eb-58f5-49b5-8a87-74126c73513f on hv-cinder-85799 Sep 17 04:38:49.594561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-d89b30da-26b8-4408-9a1c-bf19eb8e47bf req-f24715c9-b7eb-44de-8c9e-215cf683a50a service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033070 Sep 17 04:38:49.595155 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 511/1026] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:38:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:24.553283 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:24.554051 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:24.556930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:24.557768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=20 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:24.565355 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:24.565973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:24.566514 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:24.567373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:24.567937 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:24.568473 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:24.596032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-faa3ada7-0b6f-47a3-9151-72cc1da896f6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/ips" status: 200 len: 149 microversion: 2.1 time: 0.047869 Sep 17 04:39:24.596640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 516/1027] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:39:24 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/ips => generated 149 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:25.970012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=67 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:25.970820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=9 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:39:25.975269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "71b56a4f-a85b-4a15-bfd3-630eeeff614a"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:39:25.977721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:25.984732 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:25.985241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:25.985668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:25.986394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:25.986841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:25.987260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:26.265867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-259855bb-9bd2-4e1b-9633-e67302486857 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "POST /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.297727 Sep 17 04:39:26.266667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 512/1028] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:39:25 2022] POST /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments => generated 194 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:33.981391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:33.981930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:33.991626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:33.992333 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:33.992953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:33.994033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:33.994669 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:33.995278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:34.078443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'services' on Instance uuid 871c4641-f974-4cc5-bc3e-0c87446fedc6 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:39:34.165234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-d1629954-a7f6-4093-a5f9-4063b72b4f5d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments/71b56a4f-a85b-4a15-bfd3-630eeeff614a" status: 202 len: 0 microversion: 2.1 time: 0.190018 Sep 17 04:39:34.166012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 517/1029] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:39:33 2022] DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments/71b56a4f-a85b-4a15-bfd3-630eeeff614a => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:39:34.177122 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:34.179466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:34.187571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:34.188321 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:34.188767 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:34.189489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:34.189981 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:34.190385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:34.222898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-53b358de-10a6-4763-a182-6cc79deea682 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050399 Sep 17 04:39:34.223463 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 513/1030] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:39:34 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:35.242290 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:35.246123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:35.255891 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.256376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.256812 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.257550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.258005 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.258430 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.288940 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8ef48b7c-5fad-4467-8f55-135c281f6975 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055344 Sep 17 04:39:35.289534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 518/1031] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:39:35 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:39:35.354069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:35.354603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:35.361494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.361974 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.362405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.363089 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.363539 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.363955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.389133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:39:35.469929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-aed95b59-703f-4b25-844e-acd9acab9205 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 204 len: 0 microversion: 2.1 time: 0.119174 Sep 17 04:39:35.470599 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 514/1032] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:39:35 2022] DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:39:35.479819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:35.480426 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:35.486367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.487034 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.487464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.488139 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:35.488880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:35.489355 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:35.623847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:35.634479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2479491e-d30c-4909-bfa2-ad7b21247115 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1682 microversion: 2.1 time: 0.158965 Sep 17 04:39:35.635071 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 519/1033] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:35 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1682 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:39:36.665456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:36.666579 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:36.677965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:36.678523 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:36.678917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:36.679604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:36.680163 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:36.680672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:36.826913 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:36.834936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7e1d68a7-27c7-4743-842c-e8252739603c tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1682 microversion: 2.1 time: 0.177470 Sep 17 04:39:36.835509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 515/1034] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:36 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1682 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:39:37.859160 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:37.859797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:37.866643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:37.867108 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:37.867549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:37.868234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:37.868684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:37.869112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:37.999647 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:38.011732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a59d70fb-10ed-4782-b6e9-145a8957487d tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1682 microversion: 2.1 time: 0.156402 Sep 17 04:39:38.012603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 520/1035] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:37 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:39:38.891604 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "871c4641-f974-4cc5-bc3e-0c87446fedc6", "name": "network-vif-deleted", "tag": "b2b856a4-73a0-498f-8867-d7261d331366"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:39:38.902325 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:38.902934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:38.904034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:38.915557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] Creating event network-vif-deleted:b2b856a4-73a0-498f-8867-d7261d331366 for instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 on hv-cinder-85799 Sep 17 04:39:38.920701 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-18b957fc-669f-42fb-a53c-f6c737a1000d req-c45c33e3-4017-4f7b-b7fe-b6f0dc59bda7 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032806 Sep 17 04:39:38.921432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 516/1036] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:39:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:39.038412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:39.039027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:39.047127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:39.048168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:39.048830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:39.049844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:39.050367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:39.050903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:39.095596 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:39.096466 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:39.097147 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:39.097898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lazy-loading 'fault' on Instance uuid 871c4641-f974-4cc5-bc3e-0c87446fedc6 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:39:39.160780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '094abcdcd1f54a658564357056e7849a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9f593f419bb4ebeb351a9ad54a2a532', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:39.170302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1e54739f-b219-48a4-8dff-3daebb6ab646 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 200 len: 1332 microversion: 2.1 time: 0.135972 Sep 17 04:39:39.173136 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 521/1037] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:39 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 1332 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:39:40.199814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:40.200956 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:40.210040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:40.210529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:40.210968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:40.211662 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:40.212127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:40.212549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:40.226496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 HTTP/1.1 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:40.228107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:40.229899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '871c4641-f974-4cc5-bc3e-0c87446fedc6'} Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:40.230733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:40.232031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:40.233580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:40.233580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-72736d7f-be2b-4394-bb71-734dd0af9cb8 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 404 len: 111 microversion: 2.1 time: 0.037835 Sep 17 04:39:40.233580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 517/1038] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:40 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:40.238607 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c5673aa1-4671-4950-89b5-d437ff65f9bb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '.version_select at 0x7fbc4f4cbe50>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:40.258748 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c5673aa1-4671-4950-89b5-d437ff65f9bb tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2006014554" status: 202 len: 0 microversion: 2.1 time: 0.023941 Sep 17 04:39:40.259312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 522/1039] 10.222.0.60 () {58 vars in 1221 bytes} [Sat Sep 17 04:39:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2006014554 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:39:42.042676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:42.043217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:42.049853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:42.050329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:42.050757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:42.051432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:42.051878 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:42.052291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:42.067803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 HTTP/1.1 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:42.069441 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.070937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '871c4641-f974-4cc5-bc3e-0c87446fedc6'} Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:42.071971 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.073561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.075188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:42.075188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b2c17900-efdd-4faa-a569-d543df718d59 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 404 len: 111 microversion: 2.1 time: 0.032013 Sep 17 04:39:42.075188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 518/1040] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:39:42 2022] DELETE /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:42.079852 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:42.080462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 871c4641-f974-4cc5-bc3e-0c87446fedc6] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:42.087222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:42.087704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:42.088138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:42.089208 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:42.089662 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:42.090081 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:42.103245 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 HTTP/1.1 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:42.104898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.106614 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '871c4641-f974-4cc5-bc3e-0c87446fedc6'} Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:42.107529 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. Sep 17 04:39:42.109060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:42.110461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 871c4641-f974-4cc5-bc3e-0c87446fedc6 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:42.110461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ae42c5c7-b654-4ffe-9a10-1a30484fa57b tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6" status: 404 len: 111 microversion: 2.1 time: 0.030901 Sep 17 04:39:42.110461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 523/1041] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:42 2022] GET /compute/v2.1/servers/871c4641-f974-4cc5-bc3e-0c87446fedc6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:42.987530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:42.990765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:42.999247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:42.999930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.000487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.001505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.002123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.002554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.033244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-665b90f0-6d25-4fa1-9d85-d1db492e7fc8 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.052115 Sep 17 04:39:43.033791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 519/1042] 10.222.0.60 () {60 vars in 1343 bytes} [Sat Sep 17 04:39:42 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:39:43.099116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.099619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.106776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.106776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.107302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.108367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.108882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.109367 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.151824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:39:43.250532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4731fbc6-0702-462b-bf5f-13769cfcf692 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 204 len: 0 microversion: 2.20 time: 0.155157 Sep 17 04:39:43.251162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 524/1043] 10.222.0.60 () {60 vars in 1280 bytes} [Sat Sep 17 04:39:43 2022] DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 17 04:39:43.258066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.258610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.260018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.260723 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.265417 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.266086 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.266529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.266767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.267212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.267330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.267666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.267782 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.268091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.268341 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.268790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.269237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.281895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd HTTP/1.1 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:43.283469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.285205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '8bc28c77-6b66-4f4e-819e-c6047e0e6ddd'} Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:43.289339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.290821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.292314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:43.292314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c20acdcf-4e87-4f7a-b222-26da4a9349a6 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 404 len: 111 microversion: 2.1 time: 0.031502 Sep 17 04:39:43.292314 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 520/1044] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:39:43 2022] DELETE /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:43.293269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.293877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.299893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.300343 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.300797 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.301537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.302218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.302757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.316329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd HTTP/1.1 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:43.317939 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.319548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '8bc28c77-6b66-4f4e-819e-c6047e0e6ddd'} Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:43.320464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. Sep 17 04:39:43.321938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.323216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 8bc28c77-6b66-4f4e-819e-c6047e0e6ddd could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:43.323216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5fc373bb-f3c7-45bf-aeb2-926c10d1cc69 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd" status: 404 len: 111 microversion: 2.1 time: 0.029704 Sep 17 04:39:43.323216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 521/1045] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:43 2022] GET /compute/v2.1/servers/8bc28c77-6b66-4f4e-819e-c6047e0e6ddd => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:43.327495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.327990 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.333842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.334327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.334763 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.335453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.335908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.336337 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.352811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 HTTP/1.1 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:43.354393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.356071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'acb9bb23-2e07-483b-b96a-e68d1cb95835'} Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:43.356957 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.358536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.360037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:43.360037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f761fec8-455b-40ba-85fe-a2386261ab88 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 404 len: 111 microversion: 2.1 time: 0.031801 Sep 17 04:39:43.360037 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 522/1046] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:39:43 2022] DELETE /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:43.363814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:43.364503 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: acb9bb23-2e07-483b-b96a-e68d1cb95835] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:43.370727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.371218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.371649 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.372344 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:43.372791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:43.373248 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:43.386888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 HTTP/1.1 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:43.388442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.390190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'acb9bb23-2e07-483b-b96a-e68d1cb95835'} Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:43.391044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. Sep 17 04:39:43.392655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:43.393895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance acb9bb23-2e07-483b-b96a-e68d1cb95835 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:43.393895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7491d281-7d74-4728-b939-7ce0ba8419b2 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835" status: 404 len: 111 microversion: 2.1 time: 0.029508 Sep 17 04:39:43.393895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 523/1047] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:43 2022] GET /compute/v2.1/servers/acb9bb23-2e07-483b-b96a-e68d1cb95835 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 17 04:39:43.408192 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:43.416450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:43.417849 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:43.419018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-92cc3a3e-e052-4510-bb05-0e2af608a508 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.163254 Sep 17 04:39:43.419568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 525/1048] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:43 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:39:44.446635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:44.447253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:44.454841 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.455380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.455856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.456621 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.457127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.457622 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.540498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:44.541065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:44.547780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.548268 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.548703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.549434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.549875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.550439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.564745 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 HTTP/1.1 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:44.566534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.568027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '508f0590-1d24-42b5-9275-280c6cedff50'} Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:44.572351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.573792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.575308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:44.575308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2a408043-84a5-436d-948f-0aa99e63dc7e tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 404 len: 111 microversion: 2.1 time: 0.031712 Sep 17 04:39:44.575308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 526/1049] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:39:44 2022] DELETE /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:44.576713 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:44.577342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] [instance: 508f0590-1d24-42b5-9275-280c6cedff50] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:44.583728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.584286 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.584810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.585537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:44.585992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:44.586417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:44.600214 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:44.600749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] HTTP exception thrown: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Request method failure captured: Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 HTTP/1.1 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUzpNGq_ci8v5rUCc_w3J7OJETgYygRPa2ZisZiLvfCuAngNFPi-Vk8yQbq29f_Uwfns3EYAzCIAt98ixHw5HOR3xI_O6M90b7YfjfyXLXqlPt2-x3_vHqKZSh099BbQcf9UEsgCcPqascw0akFo2Y_Lqw58QD-WUsxiu86K0Php4iY Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:44.602387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e9f593f419bb4ebeb351a9ad54a2a532"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e9f593f419bb4ebeb351a9ad54a2a532 Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-376570379 Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 094abcdcd1f54a658564357056e7849a Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeNegativeTest-376570379-project Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.604021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '508f0590-1d24-42b5-9275-280c6cedff50'} Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:44.605892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. Sep 17 04:39:44.607375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:44.608833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] Returning 404 to user: Instance 508f0590-1d24-42b5-9275-280c6cedff50 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:44.608833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-59c94e34-b512-4265-bd15-ec24d91881c3 tempest-AttachVolumeNegativeTest-376570379 tempest-AttachVolumeNegativeTest-376570379-project] 10.222.0.60 "GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50" status: 404 len: 111 microversion: 2.1 time: 0.030821 Sep 17 04:39:44.608833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 527/1050] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:39:44 2022] GET /compute/v2.1/servers/508f0590-1d24-42b5-9275-280c6cedff50 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 17 04:39:44.611263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:44.612822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:44.614010 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bd4bc45f-419c-4b06-8066-8bfd395cb0ec tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.172076 Sep 17 04:39:44.614935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 524/1051] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:44 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:39:45.638698 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:45.639292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:45.646439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:45.646923 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:45.647353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:45.648047 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:45.648499 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:45.648930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:45.911012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:45.921524 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:45.922911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:45.924120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2d090fa-c9e8-4a49-b73e-5059b405b501 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1720 microversion: 2.20 time: 0.289432 Sep 17 04:39:45.924719 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 528/1052] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:45 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1720 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 17 04:39:46.860348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad101a32-7ee9-4928-b7e8-2ea39a74bf6e", "name": "network-vif-deleted", "tag": "3ab96fb5-2c5a-484d-8537-588560c96378"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:39:46.873926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:46.874574 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:46.875099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:46.888860 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] Creating event network-vif-deleted:3ab96fb5-2c5a-484d-8537-588560c96378 for instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e on hv-cinder-85799 Sep 17 04:39:46.894744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-91f765d8-1950-4019-a540-9e00b552c193 req-16fef75d-94bc-4747-8495-af2c717ada7b service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040191 Sep 17 04:39:46.896324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 525/1053] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:39:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:46.949422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:46.950202 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:46.957058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:46.957786 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:46.958523 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:46.959472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:46.960175 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:46.960811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:47.056017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:47.065450 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:47.067537 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ed4e695113f843fda52ce62792a36bb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b194b28dec41119ae0d6d913f19c31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:39:47.068898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-eaef2454-edd4-40ea-96c9-008b42aee640 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 200 len: 1365 microversion: 2.20 time: 0.123479 Sep 17 04:39:47.069875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 529/1054] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:46 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 1365 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 17 04:39:48.095837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:48.096545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:48.103510 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:48.103988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:48.104415 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:48.105076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:48.105590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:48.105987 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:48.119679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e HTTP/1.1 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:48.123570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:48.125302 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'ad101a32-7ee9-4928-b7e8-2ea39a74bf6e'} Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:48.126296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:48.127892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:48.130203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:48.131219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6a7d335a-d671-4b92-996b-df4cf13a0d13 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 404 len: 111 microversion: 2.20 time: 0.039836 Sep 17 04:39:48.131943 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 526/1055] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:48 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:39:48.140759 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3b204b7b-158c-4a58-9f6e-92b40bb13b37 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '.version_select at 0x7fbc4f549160>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:48.162602 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3b204b7b-158c-4a58-9f6e-92b40bb13b37 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-196185231" status: 204 len: 0 microversion: 2.20 time: 0.025651 Sep 17 04:39:48.163327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 530/1056] 10.222.0.60 () {60 vars in 1259 bytes} [Sat Sep 17 04:39:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-196185231 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 17 04:39:50.356877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:50.357457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:50.366777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:50.367342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:50.367842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:50.368597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:50.369137 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:50.369631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:50.383845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e HTTP/1.1 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:50.386097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.387779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'ad101a32-7ee9-4928-b7e8-2ea39a74bf6e'} Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:50.388537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.389900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.391257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:50.391257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-286ac575-3369-41dc-890f-d1b9964e61f7 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 404 len: 111 microversion: 2.20 time: 0.034698 Sep 17 04:39:50.391257 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 527/1057] 10.222.0.60 () {60 vars in 1280 bytes} [Sat Sep 17 04:39:50 2022] DELETE /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:39:50.395697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:50.396508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: ad101a32-7ee9-4928-b7e8-2ea39a74bf6e] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:50.402116 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:50.402665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:50.403031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:50.403710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:50.404161 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:50.404582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:50.418155 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e HTTP/1.1 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:50.420211 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.421994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'ad101a32-7ee9-4928-b7e8-2ea39a74bf6e'} Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:50.422779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. Sep 17 04:39:50.424051 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:50.425160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance ad101a32-7ee9-4928-b7e8-2ea39a74bf6e could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:50.425160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8f7fe0ae-e7d8-4663-a198-17af5404209f tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e" status: 404 len: 111 microversion: 2.20 time: 0.029500 Sep 17 04:39:50.425160 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 528/1058] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:50 2022] GET /compute/v2.1/servers/ad101a32-7ee9-4928-b7e8-2ea39a74bf6e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:39:50.508338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a34f2682-f3ce-4cc6-b013-8424ddc63989 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Action: 'create', calling method: .version_select at 0x7fbc4f7a1790>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1089534491"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:39:50.824452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a34f2682-f3ce-4cc6-b013-8424ddc63989 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.445150 Sep 17 04:39:50.825068 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 531/1059] 10.222.0.60 () {60 vars in 1155 bytes} [Sat Sep 17 04:39:50 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 446 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 17 04:39:50.840717 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c71f99a1-6021-4076-bf44-401255919198 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f3f0f70>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:50.889541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c71f99a1-6021-4076-bf44-401255919198 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 200 len: 345 microversion: 2.1 time: 0.052946 Sep 17 04:39:50.890074 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 529/1060] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:39:50 2022] GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 345 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:51.576097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:51.576643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:51.583833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:51.584413 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:51.584847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:51.585562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:51.586014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:51.586433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:51.601464 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 HTTP/1.1 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:51.603030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.604801 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '0043b5dd-c019-4a76-b2d0-4c68982cac21'} Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:51.605802 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.607447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.608996 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:51.608996 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0b951114-b4f7-4892-8d1a-891e6483477a tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 404 len: 111 microversion: 2.20 time: 0.032952 Sep 17 04:39:51.608996 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 532/1061] 10.222.0.60 () {60 vars in 1280 bytes} [Sat Sep 17 04:39:51 2022] DELETE /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:39:51.614458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:51.615055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] [instance: 0043b5dd-c019-4a76-b2d0-4c68982cac21] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:39:51.623586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:51.624078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:51.624522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:51.625242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:39:51.625703 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:39:51.626132 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:39:51.639404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] HTTP exception thrown: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Request method failure captured: Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 HTTP/1.1 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJUztFxbyEKEYx0NN738TyAi1uK0jQ1W4CIWyl5VJKs7XzAbnseCfh1XMzDDzYgueRYahKqBryl6eUFLOaOEPLQ_0MYQNNGqYXqdx-M0_67ojJ0lKt_BnjDRLYzkojEF6WpV_en0KUXtb7p6wMPLYqNcMQihl8INmuIthtOtPrzl_LxA Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Openstack-Nova-Api-Version: 2.20 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:51.640959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e0b194b28dec41119ae0d6d913f19c31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e0b194b28dec41119ae0d6d913f19c31 Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2033013655 Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ed4e695113f843fda52ce62792a36bb0 Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2033013655-project Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.642640 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '0043b5dd-c019-4a76-b2d0-4c68982cac21'} Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:51.643723 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. Sep 17 04:39:51.645187 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:51.651699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] Returning 404 to user: Instance 0043b5dd-c019-4a76-b2d0-4c68982cac21 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:51.651699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3ba4ff8f-a3e8-456c-9e69-75ff9c810c35 tempest-AttachVolumeShelveTestJSON-2033013655 tempest-AttachVolumeShelveTestJSON-2033013655-project] 10.222.0.60 "GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21" status: 404 len: 111 microversion: 2.20 time: 0.032468 Sep 17 04:39:51.651699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 530/1062] 10.222.0.60 () {60 vars in 1277 bytes} [Sat Sep 17 04:39:51 2022] GET /compute/v2.1/servers/0043b5dd-c019-4a76-b2d0-4c68982cac21 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 17 04:39:51.905879 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8539e5c6-e137-4b3d-9f29-4abc0a8ab389 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f557550>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:51.954651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8539e5c6-e137-4b3d-9f29-4abc0a8ab389 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 200 len: 346 microversion: 2.1 time: 0.052880 Sep 17 04:39:51.955285 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 533/1063] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:39:51 2022] GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 346 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:51.968728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7bbed6d3-04a2-4618-80c7-f1966f95ac9b tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Action: 'create', calling method: .version_select at 0x7fbc501d3820>, body: {"snapshot": {"volume_id": "14d10235-1b59-42a3-83ef-ea199bf87a19", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-153533146"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:39:52.123988 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7bbed6d3-04a2-4618-80c7-f1966f95ac9b tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.159031 Sep 17 04:39:52.125490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 531/1064] 10.222.0.60 () {60 vars in 1162 bytes} [Sat Sep 17 04:39:51 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:52.138516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4ff0ff03-0a2c-4b49-98a1-d5535ce68a65 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f4ebca0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:52.172433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4ff0ff03-0a2c-4b49-98a1-d5535ce68a65 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.039302 Sep 17 04:39:52.173687 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 534/1065] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:52 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:52.184274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a3cca7f6-32ef-4762-ae60-2cfcd0b201dd tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f3af9d0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:52.216654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a3cca7f6-32ef-4762-ae60-2cfcd0b201dd tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.036268 Sep 17 04:39:52.217346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 532/1066] 10.222.0.60 () {58 vars in 1140 bytes} [Sat Sep 17 04:39:52 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:52.228886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a69f7905-deef-45ac-98ea-01fd02a1c96e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f8000d0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:52.259790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a69f7905-deef-45ac-98ea-01fd02a1c96e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.034814 Sep 17 04:39:52.260448 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 535/1067] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:52 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:53.273506 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6a06b555-a419-48b0-8ecd-4629fd2bd872 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f60be50>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:53.303718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6a06b555-a419-48b0-8ecd-4629fd2bd872 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.034695 Sep 17 04:39:53.304356 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 533/1068] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:53 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:54.316608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-47c38a96-68d6-4e8c-b3f9-8c81313edb2f tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f557a60>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:54.348483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-47c38a96-68d6-4e8c-b3f9-8c81313edb2f tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.036319 Sep 17 04:39:54.349058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 536/1069] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:54 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:55.364726 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cd47fbbb-b376-4a95-81cd-7f9288260b9d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f3f0a60>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:55.397800 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cd47fbbb-b376-4a95-81cd-7f9288260b9d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.037492 Sep 17 04:39:55.398375 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 534/1070] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:55 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:56.411455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8a2b0f5e-c980-44b2-9fc8-7bcde27cabbb tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f800a60>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:56.442709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8a2b0f5e-c980-44b2-9fc8-7bcde27cabbb tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.035825 Sep 17 04:39:56.443382 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 537/1071] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:56 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:57.455583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-26d9a267-c8bb-47e6-9532-e87157cb0b9c tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f39a310>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:57.492400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-26d9a267-c8bb-47e6-9532-e87157cb0b9c tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 287 microversion: 2.1 time: 0.040581 Sep 17 04:39:57.492974 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 535/1072] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:57 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 287 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:57.504980 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e1a546a6-93b5-46da-ad0d-3d922e38b892 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f9060d0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:57.574670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e1a546a6-93b5-46da-ad0d-3d922e38b892 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "DELETE /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 202 len: 0 microversion: 2.1 time: 0.074112 Sep 17 04:39:57.575308 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 538/1073] 10.222.0.60 () {58 vars in 1254 bytes} [Sat Sep 17 04:39:57 2022] DELETE /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:39:57.584411 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-65eb0785-87c9-4b8b-a7d4-2d843f8acc0c tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f39a4c0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:57.631754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-65eb0785-87c9-4b8b-a7d4-2d843f8acc0c tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 200 len: 286 microversion: 2.1 time: 0.051863 Sep 17 04:39:57.639512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 536/1074] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:57 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 286 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:58.647984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3ab3bbba-bd86-4247-a830-d6efa47af6ce tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f56c4c0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:58.692741 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-3ab3bbba-bd86-4247-a830-d6efa47af6ce tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] HTTP exception thrown: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3ab3bbba-bd86-4247-a830-d6efa47af6ce tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Request method failure captured: Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 HTTP/1.1 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU-WbtvDjuDgjfvdJLXwW39HIuI--brWhLnRFs__j_tOMnPTd1_4ShSn_li-2QI_MPcIGrabolRfXevSFNujDcBgyVsgDtSI6b5XLKk7AIcJMCMrYeQGzyBztBD674PZtc5kyEWsgZyr9nIwFDaja0SqwmHsIc5qPTmYGxvIiJNnGl8 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:58.695124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 70bb0b4438f845a998daeafb85504dec Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '33312972-f2d8-4c0c-850b-c4c32b5387c8'} Sep 17 04:39:58.696835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. (HTTP 404) (Request-ID: req-959da18c-dce7-4621-b2f5-06b7cea4249f) Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.697877 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:39:58.699099 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 17 04:39:58.700724 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3ab3bbba-bd86-4247-a830-d6efa47af6ce tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Returning 404 to user: Snapshot 33312972-f2d8-4c0c-850b-c4c32b5387c8 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3ab3bbba-bd86-4247-a830-d6efa47af6ce tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8" status: 404 len: 111 microversion: 2.1 time: 0.053776 Sep 17 04:39:58.702277 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 539/1075] 10.222.0.60 () {58 vars in 1251 bytes} [Sat Sep 17 04:39:58 2022] GET /compute/v2.1/os-snapshots/33312972-f2d8-4c0c-850b-c4c32b5387c8 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:58.705457 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-974c3a1a-6eaa-4e85-8821-ed78ed93b7e5 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f4834c0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:58.785381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-974c3a1a-6eaa-4e85-8821-ed78ed93b7e5 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 202 len: 0 microversion: 2.1 time: 0.083618 Sep 17 04:39:58.786453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 537/1076] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:39:58 2022] DELETE /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:39:58.795176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-31c31763-7e26-4924-9bc1-45527a8d010a tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f2fe700>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:58.865488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-31c31763-7e26-4924-9bc1-45527a8d010a tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 200 len: 345 microversion: 2.1 time: 0.074165 Sep 17 04:39:58.866066 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 540/1077] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:39:58 2022] GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 345 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:39:59.883803 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-87d8d3f6-d7ba-4f7b-afb0-c3c6f813219e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f3ba310>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:59.915774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-87d8d3f6-d7ba-4f7b-afb0-c3c6f813219e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] HTTP exception thrown: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-87d8d3f6-d7ba-4f7b-afb0-c3c6f813219e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Request method failure captured: Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 HTTP/1.1 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU-WbtvDjuDgjfvdJLXwW39HIuI--brWhLnRFs__j_tOMnPTd1_4ShSn_li-2QI_MPcIGrabolRfXevSFNujDcBgyVsgDtSI6b5XLKk7AIcJMCMrYeQGzyBztBD674PZtc5kyEWsgZyr9nIwFDaja0SqwmHsIc5qPTmYGxvIiJNnGl8 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:39:59.917936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 70bb0b4438f845a998daeafb85504dec Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '14d10235-1b59-42a3-83ef-ea199bf87a19'} Sep 17 04:39:59.919397 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. (HTTP 404) (Request-ID: req-aa95f150-d5f0-4fb3-a9ce-b998e7741bd4) Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.920239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:39:59.921886 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 17 04:39:59.923293 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-87d8d3f6-d7ba-4f7b-afb0-c3c6f813219e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Returning 404 to user: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-87d8d3f6-d7ba-4f7b-afb0-c3c6f813219e tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 404 len: 109 microversion: 2.1 time: 0.040877 Sep 17 04:39:59.924456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 538/1078] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:39:59 2022] GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:39:59.935598 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fd00a814-fed7-40d4-aafb-a73cff05e17d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f559ee0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:59.966015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-fd00a814-fed7-40d4-aafb-a73cff05e17d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] HTTP exception thrown: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fd00a814-fed7-40d4-aafb-a73cff05e17d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Request method failure captured: Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 HTTP/1.1 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU-WbtvDjuDgjfvdJLXwW39HIuI--brWhLnRFs__j_tOMnPTd1_4ShSn_li-2QI_MPcIGrabolRfXevSFNujDcBgyVsgDtSI6b5XLKk7AIcJMCMrYeQGzyBztBD674PZtc5kyEWsgZyr9nIwFDaja0SqwmHsIc5qPTmYGxvIiJNnGl8 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:39:59.968048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: 70bb0b4438f845a998daeafb85504dec Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '14d10235-1b59-42a3-83ef-ea199bf87a19'} Sep 17 04:39:59.969811 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. (HTTP 404) (Request-ID: req-be503180-f68f-46ff-923f-e3519be1fe7c) Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.970672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 17 04:39:59.972247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 17 04:39:59.975472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fd00a814-fed7-40d4-aafb-a73cff05e17d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Returning 404 to user: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:39:59.978143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fd00a814-fed7-40d4-aafb-a73cff05e17d tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "DELETE /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 404 len: 109 microversion: 2.1 time: 0.038235 Sep 17 04:39:59.979881 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3500e8b9-cef5-45d1-a8c7-622903d84115 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Calling method '.version_select at 0x7fbc4f3e3f70>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:39:59.983003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 541/1079] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:39:59 2022] DELETE /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:40:00.008707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-3500e8b9-cef5-45d1-a8c7-622903d84115 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] HTTP exception thrown: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3500e8b9-cef5-45d1-a8c7-622903d84115 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Request method failure captured: Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 HTTP/1.1 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU-WbtvDjuDgjfvdJLXwW39HIuI--brWhLnRFs__j_tOMnPTd1_4ShSn_li-2QI_MPcIGrabolRfXevSFNujDcBgyVsgDtSI6b5XLKk7AIcJMCMrYeQGzyBztBD674PZtc5kyEWsgZyr9nIwFDaja0SqwmHsIc5qPTmYGxvIiJNnGl8 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:40:00.011049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/f5eef26df9ed4b079274385534f4ec40"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: f5eef26df9ed4b079274385534f4ec40 Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-5502381 Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: 70bb0b4438f845a998daeafb85504dec Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-VolumesSnapshotsTestJSON-5502381-project Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '14d10235-1b59-42a3-83ef-ea199bf87a19'} Sep 17 04:40:00.012608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. (HTTP 404) (Request-ID: req-af557b12-799a-40b2-a2a6-9b1237dc2946) Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:40:00.013628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 17 04:40:00.015029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 17 04:40:00.016377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3500e8b9-cef5-45d1-a8c7-622903d84115 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] Returning 404 to user: Volume 14d10235-1b59-42a3-83ef-ea199bf87a19 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:40:00.018607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3500e8b9-cef5-45d1-a8c7-622903d84115 tempest-VolumesSnapshotsTestJSON-5502381 tempest-VolumesSnapshotsTestJSON-5502381-project] 10.222.0.60 "GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19" status: 404 len: 109 microversion: 2.1 time: 0.043317 Sep 17 04:40:00.023608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 539/1080] 10.222.0.60 () {58 vars in 1245 bytes} [Sat Sep 17 04:39:59 2022] GET /compute/v2.1/os-volumes/14d10235-1b59-42a3-83ef-ea199bf87a19 => generated 109 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:40:00.765502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e711ddc8-be5b-4df5-b080-a857f26ff050 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: .version_select at 0x7fbc4f557040>, body: {"keypair": {"name": "tempest-keypair-1609495567", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCOOBP7kx1VJ243AlKN0D+hrNS0Mk45R4dlxrHpf5Ec6RtlMCmIFYN/qHpinvRHtyZJzG7pg2bfVU4AeGsfWE9Nhuh4l9BF2siyQqKSdyGZce9C2JGlY/cVg6CsE3rl2GA=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:40:00.771880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:00.792154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-e711ddc8-be5b-4df5-b080-a857f26ff050 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for project e4563767772c486f8fbb9ea131771412. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:40:00.799267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-e711ddc8-be5b-4df5-b080-a857f26ff050 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for user a486a4bb6b724dacbb78a9e0fe6a789e and project e4563767772c486f8fbb9ea131771412. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:40:00.823731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e711ddc8-be5b-4df5-b080-a857f26ff050 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.236672 Sep 17 04:40:00.824921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 542/1081] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:40:00 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:40:02.294104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:02.294869 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:02.299968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1280965222", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "276b59e3-5161-4328-9145-0306fa8cd354"}], "security_groups": [{"name": "tempest-securitygroup--888954785"}], "key_name": "tempest-keypair-1609495567", "user_data": "}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:40:02.417100 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] validate_networks() for [('276b59e3-5161-4328-9145-0306fa8cd354', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:40:02.829688 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:40:02.830112 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:40:02.830587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:40:02.830993 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:40:02.831412 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:40:02.840748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:02.841269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:02.841700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:02.842418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:02.842873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:02.843295 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:02.860709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for project e4563767772c486f8fbb9ea131771412. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:40:02.864591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for user a486a4bb6b724dacbb78a9e0fe6a789e and project e4563767772c486f8fbb9ea131771412. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:40:02.875779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:40:02.878175 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:40:02.931769 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-89cc21da-cdcc-48c9-88cb-74735f631cf7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.638849 Sep 17 04:40:02.932833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 540/1082] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:40:02 2022] POST /compute/v2.1/servers => generated 401 bytes in 640 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:40:02.943810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aebbff95-50d1-4c59-8f20-7037c10b1777 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:02.944583 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aebbff95-50d1-4c59-8f20-7037c10b1777 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:02.947029 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=79,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:03.006025 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aebbff95-50d1-4c59-8f20-7037c10b1777 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:03.012977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aebbff95-50d1-4c59-8f20-7037c10b1777 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1238 microversion: 2.1 time: 0.073108 Sep 17 04:40:03.013762 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 543/1083] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:02 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1238 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:40:04.037475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:04.038089 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:04.046931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:04.047721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:04.048400 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:04.049352 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:04.049994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:04.050890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:04.126709 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:04.136033 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2fab4b85-296a-4b71-86b1-812f35302658 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1290 microversion: 2.1 time: 0.103968 Sep 17 04:40:04.138502 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 541/1084] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:04 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1290 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:05.161671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:05.163253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:05.170562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:05.171057 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:05.171540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:05.172305 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:05.172788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:05.173323 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:05.427415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:05.436375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-43d1d9e2-bc15-4e34-a5e7-915a12008199 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.279551 Sep 17 04:40:05.437261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 544/1085] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:05 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:05.871655 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2113221b-5936-4b6e-8e99-2dc1b3ab6815", "tag": "4b75ce58-1ab1-41d9-9d19-7da81d3db517"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:40:05.887071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:05.887560 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:05.888195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:05.905796 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] Creating event network-changed:4b75ce58-1ab1-41d9-9d19-7da81d3db517 for instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 on hv-cinder-85799 Sep 17 04:40:05.914223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-b20bbb0e-4ed3-411c-94e6-e265e590d7da req-9b017606-437a-4355-93d3-bb8f3ef11568 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045575 Sep 17 04:40:05.915144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 542/1086] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:40:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:40:06.462406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:06.463030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:06.470997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:06.471558 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:06.472058 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:06.472845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:06.473406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:06.473872 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:06.615166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:06.623380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5e45e882-cf53-4e60-8829-1ba140105848 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.166146 Sep 17 04:40:06.623941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 545/1087] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:06 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:07.652208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:07.653288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:07.663309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:07.663775 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:07.664242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:07.664926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:07.665416 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:07.665813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:07.798232 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:07.807349 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dc87031f-7090-4a08-9fda-4ce1a70a2f66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.163327 Sep 17 04:40:07.807925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 543/1088] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:07 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:08.830107 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:08.830689 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:08.837984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:08.838840 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:08.839281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:08.839971 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:08.840412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:08.840841 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:08.969248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:08.977311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1f4aaaa0-443f-41a5-b387-33fd29199304 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.151153 Sep 17 04:40:08.977878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 546/1089] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:08 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:10.004405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:10.005277 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:10.012500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:10.012986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:10.013447 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:10.014139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:10.014594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:10.015020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:10.301101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:10.308933 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0ba714a8-6bbb-40cc-a4b3-9a52b8b1dd90 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.311523 Sep 17 04:40:10.309571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 544/1090] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:09 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:11.331770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:11.332957 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:11.340569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:11.340921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:11.341383 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:11.342080 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:11.342534 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:11.342959 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:11.474696 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:11.485651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0c999a7a-d66d-4cbf-82cb-03e8ca38c7cd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.154840 Sep 17 04:40:11.485651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 547/1091] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:11 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:12.508089 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:12.508719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:12.515898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:12.516387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:12.516818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:12.517549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:12.518002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:12.518451 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:12.648529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:12.656768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2b295231-31a7-4b42-931b-4797e9400d31 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.153692 Sep 17 04:40:12.658099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 545/1092] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:12 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:13.681015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:13.681732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:13.684229 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:13.690688 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:13.691217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:13.691659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:13.692383 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:13.692865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:13.693370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:13.830183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:13.838848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-90e76eaf-698b-4798-bf64-520ee9bf4ed8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.161716 Sep 17 04:40:13.839418 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 548/1093] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:13 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:14.862910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:14.863941 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:14.867471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:40:14.867798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:40:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 (ip 10.222.0.60) !!! Sep 17 04:40:14.868731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:14.869580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:40:14.879349 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:14.880169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:14.880777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:14.881946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:14.883004 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:14.883710 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:15.061602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:15.069296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3281a54e-7942-4c13-b6b9-7528e32182c7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.211570 Sep 17 04:40:15.069853 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 546/1094] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:14 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:16.092527 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:16.093137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:16.099547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:16.100018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:16.100478 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:16.101186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:16.101637 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:16.102037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:16.234417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:16.244654 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1df2aa37-97d0-4e03-baec-31bfdd78d532 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.156404 Sep 17 04:40:16.257481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 549/1095] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:16 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:16.587558 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2113221b-5936-4b6e-8e99-2dc1b3ab6815", "name": "network-vif-plugged", "status": "completed", "tag": "4b75ce58-1ab1-41d9-9d19-7da81d3db517"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:40:16.597260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:16.597856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:16.598292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:16.609031 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] Creating event network-vif-plugged:4b75ce58-1ab1-41d9-9d19-7da81d3db517 for instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 on hv-cinder-85799 Sep 17 04:40:16.614607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-b3db12f5-44f4-4eef-a2ee-199f9f14279b req-66f24cc9-a5ac-438a-a17e-0d6ea317b6d0 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.214548 Sep 17 04:40:16.615289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 547/1096] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:40:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:40:17.271460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:17.272485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:17.283412 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:17.284180 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:17.284898 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:17.286086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:17.286834 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:17.287563 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:17.423244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:17.430947 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fb2b8a09-2684-4c86-89eb-a3ce6f3bdd8b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1363 microversion: 2.1 time: 0.166215 Sep 17 04:40:17.431515 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 550/1097] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:17 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1363 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:18.454263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:18.455256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:18.461752 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:18.462272 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:18.462733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:18.463816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:18.464282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:18.464729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:18.600787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:40:18.608875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-741b76c2-1184-4ba5-af52-a48719f0b3fc tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1547 microversion: 2.1 time: 0.159822 Sep 17 04:40:18.616669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 548/1098] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:40:18 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1547 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:40:18.631342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:40:18.631830 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:40:18.638248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:18.638733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:18.639165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:18.639856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:18.640309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:18.640789 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:18.727334 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-58015c50-4ed7-478e-9ed3-794085e9e218 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099828 Sep 17 04:40:18.727855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 551/1099] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:40:18 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:40:20.110672 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2113221b-5936-4b6e-8e99-2dc1b3ab6815", "tag": "4b75ce58-1ab1-41d9-9d19-7da81d3db517"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:40:20.121681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:40:20.122162 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:40:20.122753 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:40:20.135034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] Creating event network-changed:4b75ce58-1ab1-41d9-9d19-7da81d3db517 for instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 on hv-cinder-85799 Sep 17 04:40:20.140829 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-122b68f9-8bc3-41c4-96e9-2b2db6d15fd9 req-84908fc1-206e-41a7-8d3a-d601fc2f70c4 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.181815 Sep 17 04:40:20.141728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 549/1100] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:40:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:04.942344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:41:04.942977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:41:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f (ip 10.222.0.60) !!! Sep 17 04:41:04.943809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:04.945069 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=46 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:04.948801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bc4b19eb-58f5-49b5-8a87-74126c73513f", "tag": "db2efe3b-92b0-4a7f-9c0c-416d334db906"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:04.949854 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:04.950357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:04.957994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:04.958650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=44 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:04.958984 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:04.959781 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:04.960455 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:04.961225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:04.961881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:04.962360 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:04.964768 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:04.965290 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:04.965735 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:04.981494 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] Creating event network-changed:db2efe3b-92b0-4a7f-9c0c-416d334db906 for instance bc4b19eb-58f5-49b5-8a87-74126c73513f on hv-cinder-85799 Sep 17 04:41:04.987068 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-b53d8e80-b40c-4d90-82e1-6fc5ecd32bf8 req-f839f3fe-8cc4-4f89-9585-1bf7f70d3012 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043546 Sep 17 04:41:04.987669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 550/1101] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:41:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:04.992968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:41:05.080732 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0b4c9b08-9650-4a83-bb2c-66e163a02077 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "DELETE /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 204 len: 0 microversion: 2.1 time: 0.139735 Sep 17 04:41:05.082998 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 552/1102] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:41:04 2022] DELETE /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:41:05.092006 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:05.093182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:05.100854 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:05.101372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:05.102003 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:05.102926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:05.103346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:05.103779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:05.494799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:05.503081 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7617e381-be02-4f79-b503-fe74284a11eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1578 microversion: 2.1 time: 0.416724 Sep 17 04:41:05.503877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 551/1103] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:05 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1578 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:06.527737 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:06.528479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:06.535302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:06.535932 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:06.536373 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:06.537063 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:06.537722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:06.538311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:06.764946 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:06.773274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-89c26ee8-b7c7-43c0-adad-0570ccb844ff tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1578 microversion: 2.1 time: 0.249644 Sep 17 04:41:06.773866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 553/1104] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:06 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1578 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:07.797624 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:07.798489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:07.805554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:07.806139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:07.806581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:07.807291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:07.807749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:07.808176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:07.940501 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:07.948578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-675f0b60-462e-4a87-9712-1f3ebd97bcd5 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1459 microversion: 2.1 time: 0.155121 Sep 17 04:41:07.949176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 552/1105] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:07 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1459 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:08.450650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bc4b19eb-58f5-49b5-8a87-74126c73513f", "name": "network-vif-deleted", "tag": "db2efe3b-92b0-4a7f-9c0c-416d334db906"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:08.461751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:08.462233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:08.463031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:08.475121 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] Creating event network-vif-deleted:db2efe3b-92b0-4a7f-9c0c-416d334db906 for instance bc4b19eb-58f5-49b5-8a87-74126c73513f on hv-cinder-85799 Sep 17 04:41:08.480213 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-56b20675-c92b-43a4-9a67-a8210c71afc9 req-4ec294a3-9200-4a98-b6bf-52aa144dbe21 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033491 Sep 17 04:41:08.481297 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 554/1106] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:41:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:08.973897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:08.974472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:08.981529 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:08.982142 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:08.982586 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:08.983289 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:08.983747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:08.984206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:09.016094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:09.016721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:09.017332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:09.017780 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lazy-loading 'fault' on Instance uuid bc4b19eb-58f5-49b5-8a87-74126c73513f {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:41:09.059080 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:09.068953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f0f3bdb0-8850-4056-805b-789006287a0c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 200 len: 1230 microversion: 2.1 time: 0.099989 Sep 17 04:41:09.069516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 553/1107] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:08 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 1230 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:10.097715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:10.098848 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: bc4b19eb-58f5-49b5-8a87-74126c73513f] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:10.105878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:10.106354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:10.106780 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:10.107485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:10.107928 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:10.108356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:10.121875 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] HTTP exception thrown: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Request method failure captured: Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f HTTP/1.1 Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU10aVklLgmHscuVU-P_qRXWpxyIi6BuYozKUiz_g1RNdopf3m3ZxkfLshUkyOZN1IrJrzlPmza_SVVEeFGpA58DRH_yMJIW0H3GhBIg4U_sVvt42SOSveNU04ThmFoyxguY_L6uEo_JnxLDGBqDu_NVF8ThtqeHvY2olc3IJ48zI4o Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:41:10.123539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: ff241cbd3f7c47328ac5ef2ed4c66e18 Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. Sep 17 04:41:10.125223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': 'bc4b19eb-58f5-49b5-8a87-74126c73513f'} Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:41:10.126045 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. Sep 17 04:41:10.127568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:10.129075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Returning 404 to user: Instance bc4b19eb-58f5-49b5-8a87-74126c73513f could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:41:10.129075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2358ab3e-c1be-461f-83ae-233eafa97dbf tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f" status: 404 len: 111 microversion: 2.1 time: 0.035104 Sep 17 04:41:10.129075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 555/1108] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:10 2022] GET /compute/v2.1/servers/bc4b19eb-58f5-49b5-8a87-74126c73513f => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:41:13.262771 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:13.264559 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:13.265181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:13.271512 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:13.272034 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:13.272467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:13.273221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:13.273670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:13.274127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:13.285641 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:13.286125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:13.286594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:13.298040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] Creating event network-changed:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:41:13.303092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:41:13.303663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-a14105c1-645b-456f-a6b4-b34f95ccee06 req-d04632da-6014-4558-8421-a6e8cf490113 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045119 Sep 17 04:41:13.304772 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 556/1109] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:41:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:13.388236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-6cefab0f-d9fb-48f9-a6ae-45046b928791 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "DELETE /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 204 len: 0 microversion: 2.1 time: 0.135000 Sep 17 04:41:13.389002 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 554/1110] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:41:13 2022] DELETE /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:41:13.398611 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:13.399399 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:13.405755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:13.406476 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:13.406912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:13.407608 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:13.408247 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:13.408674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:13.550770 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:13.559031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f1293c3d-7069-4917-910f-aaa78eac44ce tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.165068 Sep 17 04:41:13.559606 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 557/1111] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:13 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:14.588842 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:14.589937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:14.600835 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:14.601705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:14.602487 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:14.603191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:14.603650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:14.604076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:14.743619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:14.752241 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-41c77ffe-3dd0-4598-93c5-cc68e0118457 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.171292 Sep 17 04:41:14.752813 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 555/1112] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:14 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:15.782194 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:15.783241 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:15.794440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:15.795229 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:15.795956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:15.797114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:15.797884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:15.798590 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:15.936116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:15.947233 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9b7f3ee4-c067-4857-8307-dcfe0d9e311f tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.171409 Sep 17 04:41:15.948027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 558/1113] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:15 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:16.980107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:16.980686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:16.987705 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:16.988190 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:16.988706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:16.989436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:16.989891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:16.990313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:17.126994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:17.135245 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-dd2f5be5-638e-4c27-968c-47926f5a5d58 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.160122 Sep 17 04:41:17.135783 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 556/1114] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:16 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:18.163291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:18.164397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:18.175225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:18.176037 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:18.176761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:18.177956 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:18.178714 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:18.179429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:18.329774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:18.341792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ef691c02-7b32-4022-9e17-0e58a0175935 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.185027 Sep 17 04:41:18.342613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 559/1115] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:18 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:18.617626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "name": "network-vif-unplugged", "status": "completed", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:18.628063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:18.628620 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:18.629046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:18.640809 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] Creating event network-vif-unplugged:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:41:18.646528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-66e747a9-8c88-4762-be10-24a2dec79f0d req-62f18d4d-7cd0-44ac-8968-fe72c0e5196a service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032975 Sep 17 04:41:18.646528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 557/1116] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:41:18 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:19.369462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:19.370353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:19.381249 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:19.382050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:19.382760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:19.383887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:19.384354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:19.384753 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:19.527263 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:19.535166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-aa120ab8-bd04-4571-a98f-45d13de9d965 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.172460 Sep 17 04:41:19.535736 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 560/1117] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:19 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:20.563626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:20.564830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:20.574181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:20.574665 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:20.575095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:20.575778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:20.576230 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:20.576645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:20.722838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:20.730777 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4b43538b-5afe-4059-bcc1-53526715980b tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.174349 Sep 17 04:41:20.731313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 558/1118] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:20 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:21.754508 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:21.755125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:21.762105 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:21.762593 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:21.763032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:21.763725 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:21.764190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:21.764619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:21.904303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:21.915751 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-582b175c-47ac-4e25-b1eb-42df8b36fc26 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1577 microversion: 2.1 time: 0.165173 Sep 17 04:41:21.916487 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 561/1119] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:21 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1577 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:22.938991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:22.939482 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:22.946693 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:22.947193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:22.947724 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:22.948377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:22.948827 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:22.949280 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:23.018018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff241cbd3f7c47328ac5ef2ed4c66e18', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '752e65efe679458fab6b8dbc3a9de82f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:41:23.026097 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7a1cbb5d-e34e-422b-a416-bdd7c5085e53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 200 len: 1391 microversion: 2.1 time: 0.091023 Sep 17 04:41:23.026786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 559/1120] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:22 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 1391 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:41:23.083328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1e695be-8ee8-42b5-88b0-79d0836d8fbf", "name": "network-vif-deleted", "tag": "35e61da4-2924-436c-a984-9952b3c29f0a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:23.096815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:23.097581 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:23.098230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:23.115652 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] Creating event network-vif-deleted:35e61da4-2924-436c-a984-9952b3c29f0a for instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf on hv-cinder-85799 Sep 17 04:41:23.122917 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-a31d59bc-710c-4a0b-98a9-5089f06f7296 req-3a660ddb-8caf-4f79-8d75-d9a722b85e40 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.171384 Sep 17 04:41:23.123817 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 562/1121] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:41:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:41:24.054615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:24.055578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: a1e695be-8ee8-42b5-88b0-79d0836d8fbf] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:24.063198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.063681 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.064110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.064798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.065292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.065708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.079072 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] HTTP exception thrown: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Request method failure captured: Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf HTTP/1.1 Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU10aVklLgmHscuVU-P_qRXWpxyIi6BuYozKUiz_g1RNdopf3m3ZxkfLshUkyOZN1IrJrzlPmza_SVVEeFGpA58DRH_yMJIW0H3GhBIg4U_sVvt42SOSveNU04ThmFoyxguY_L6uEo_JnxLDGBqDu_NVF8ThtqeHvY2olc3IJ48zI4o Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:41:24.080700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ff241cbd3f7c47328ac5ef2ed4c66e18 Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. Sep 17 04:41:24.082543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': 'a1e695be-8ee8-42b5-88b0-79d0836d8fbf'} Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:41:24.083500 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. Sep 17 04:41:24.085102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.086496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Returning 404 to user: Instance a1e695be-8ee8-42b5-88b0-79d0836d8fbf could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:41:24.086496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-61926873-2e95-44f4-b8e0-ff39ca74a995 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf" status: 404 len: 111 microversion: 2.1 time: 0.034060 Sep 17 04:41:24.086496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 560/1122] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:24 2022] GET /compute/v2.1/servers/a1e695be-8ee8-42b5-88b0-79d0836d8fbf => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:41:24.543318 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:24.543899 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:24.550960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.551447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.551888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.552562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.553043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.553497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.567196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] HTTP exception thrown: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Request method failure captured: Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 HTTP/1.1 Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU10aVklLgmHscuVU-P_qRXWpxyIi6BuYozKUiz_g1RNdopf3m3ZxkfLshUkyOZN1IrJrzlPmza_SVVEeFGpA58DRH_yMJIW0H3GhBIg4U_sVvt42SOSveNU04ThmFoyxguY_L6uEo_JnxLDGBqDu_NVF8ThtqeHvY2olc3IJ48zI4o Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:41:24.568878 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: ff241cbd3f7c47328ac5ef2ed4c66e18 Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.570556 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '8f28c067-5b02-4cb7-8284-c59d2f694528'} Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:41:24.571467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.572881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.574181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Returning 404 to user: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:41:24.574181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a1a92627-bb7b-498a-9c85-6054535aa00c tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "DELETE /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 404 len: 111 microversion: 2.1 time: 0.031799 Sep 17 04:41:24.574181 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 563/1123] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:41:24 2022] DELETE /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:41:24.579514 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:24.580133 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] [instance: 8f28c067-5b02-4cb7-8284-c59d2f694528] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:24.586267 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.586722 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.587155 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.588201 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:24.588750 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:24.589203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:24.602821 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] HTTP exception thrown: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Request method failure captured: Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 HTTP/1.1 Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU10aVklLgmHscuVU-P_qRXWpxyIi6BuYozKUiz_g1RNdopf3m3ZxkfLshUkyOZN1IrJrzlPmza_SVVEeFGpA58DRH_yMJIW0H3GhBIg4U_sVvt42SOSveNU04ThmFoyxguY_L6uEo_JnxLDGBqDu_NVF8ThtqeHvY2olc3IJ48zI4o Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:41:24.604351 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/752e65efe679458fab6b8dbc3a9de82f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: 752e65efe679458fab6b8dbc3a9de82f Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-TestVolumeBootPattern-1059542158 Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: ff241cbd3f7c47328ac5ef2ed4c66e18 Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-TestVolumeBootPattern-1059542158-project Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.606012 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '8f28c067-5b02-4cb7-8284-c59d2f694528'} Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:41:24.606885 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. Sep 17 04:41:24.608438 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:41:24.610083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Returning 404 to user: Instance 8f28c067-5b02-4cb7-8284-c59d2f694528 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:41:24.610083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-7f88eb38-827e-466a-9de6-5bb3c81a63eb tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528" status: 404 len: 111 microversion: 2.1 time: 0.030106 Sep 17 04:41:24.610083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 561/1124] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:41:24 2022] GET /compute/v2.1/servers/8f28c067-5b02-4cb7-8284-c59d2f694528 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:41:25.933261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0ac41c91-7446-4c51-9d28-02d166217a53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] Calling method '.version_select at 0x7fbc4f5599d0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:25.953856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0ac41c91-7446-4c51-9d28-02d166217a53 tempest-TestVolumeBootPattern-1059542158 tempest-TestVolumeBootPattern-1059542158-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-623666681" status: 202 len: 0 microversion: 2.1 time: 0.025322 Sep 17 04:41:25.954483 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 564/1125] 10.222.0.60 () {58 vars in 1260 bytes} [Sat Sep 17 04:41:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-623666681 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:41:49.713587 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:41:49.714174 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:49.716946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=31,nova_api:UPDATE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:49.717659 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:49.724661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:49.725216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:49.725664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:49.726354 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:49.726811 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:49.727243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:49.752626 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-fe46ab0f-07c9-4bff-ade2-eacc7a787f23 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/ips" status: 200 len: 144 microversion: 2.1 time: 0.043374 Sep 17 04:41:49.753221 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 562/1126] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:41:49 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/ips => generated 144 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:41:52.398382 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:52.399237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:41:52.403470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "15d97420-8110-4598-9f0a-e5195077f493"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:41:52.406729 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:41:52.414609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:52.415086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:52.415518 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:52.416207 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:41:52.416647 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:41:52.417072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:41:52.707704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7e470fe7-40d9-4fbe-97e9-1cc4ffc0d637 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.310468 Sep 17 04:41:52.708307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 565/1127] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:41:52 2022] POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments => generated 194 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:42:00.333024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:42:00.335544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:42:00.336191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=4 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:42:00.343673 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:00.344158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:00.344594 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:00.403556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Going to try to stop instance {{(pid=120627) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 17 04:42:00.454837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-30835f3c-1e50-443c-8a82-040acdfcb180 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action" status: 202 len: 0 microversion: 2.1 time: 0.124854 Sep 17 04:42:00.455581 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 563/1128] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:42:00 2022] POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:42:00.466088 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:00.466874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:00.473231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:00.473707 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:00.474131 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:00.474846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:00.475302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:00.475720 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:00.619888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:00.628870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-55759396-8f83-4d23-b291-5a298a44ef2a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.168391 Sep 17 04:42:00.629547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 566/1129] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:00 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:01.656609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:01.657544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:01.665812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:01.666427 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:01.666977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:01.667852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:01.668423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:01.668958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:01.804544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:01.812859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-990e4b79-dcc9-4277-9836-955b123f6310 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.162832 Sep 17 04:42:01.813483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 564/1130] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:01 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:02.840570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:02.841274 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:02.849469 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:02.850040 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:02.850552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:02.851371 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:02.851903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:02.852407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:03.008117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:03.021502 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c6157f80-ef44-42c3-a488-b64f866ecdff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.187740 Sep 17 04:42:03.022292 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 567/1131] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:02 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:04.050760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:04.051814 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:04.062725 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:04.063518 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:04.064229 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:04.065414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:04.066175 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:04.066877 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:04.201143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:04.210111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-435f74e3-1c35-4081-8561-8a87b3239e29 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.165861 Sep 17 04:42:04.210689 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 565/1132] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:04 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:05.235199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:05.235889 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:05.242805 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:05.243295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:05.243725 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:05.244421 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:05.244881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:05.245357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:05.378345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:05.387082 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f64ef879-6dfe-45af-92a9-15fc822eb279 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.155760 Sep 17 04:42:05.387658 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 568/1133] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:05 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:06.413685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:06.414643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:06.425547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:06.426355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:06.427078 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:06.428256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:06.428975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:06.429745 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:06.561288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:06.574719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-eeec8635-b50a-48c4-a57a-b93f3e5e5d4b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.167371 Sep 17 04:42:06.575424 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 566/1134] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:06 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:07.602799 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:07.603796 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:07.611981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:07.612475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:07.612912 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:07.613664 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:07.614117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:07.614543 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:07.747300 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:07.756230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e889a942-3447-4570-9312-884410103ff4 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.160129 Sep 17 04:42:07.756815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 569/1135] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:07 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:08.791464 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:08.792471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:08.800708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:08.801249 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:08.801682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:08.802381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:08.802838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:08.803239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:09.186955 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:09.195580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f984d285-1dd7-4be0-be7d-d80fc97994d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.410846 Sep 17 04:42:09.196155 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 567/1136] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:08 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:10.223153 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:10.224117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:10.234888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:10.235671 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:10.236387 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:10.237605 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:10.238354 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:10.239054 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:10.379215 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:10.387853 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-11bded8a-d2fe-4693-8d2f-07d093d29596 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.171379 Sep 17 04:42:10.388594 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 570/1137] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:10 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:11.415027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:11.415965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:11.427699 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:11.428466 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:11.429237 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:11.430382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:11.431139 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:11.431845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:11.573236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:11.585936 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f6141beb-786c-44a8-8f48-e46c66bd508a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.177328 Sep 17 04:42:11.587071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 568/1138] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:11 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:12.611018 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:12.611882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:12.622363 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:12.623198 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:12.623893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:12.625166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:12.625913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:12.626634 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:12.761239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:12.769972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-61c19207-d662-4b2b-8f74-687a44959f61 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.164161 Sep 17 04:42:12.770541 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 571/1139] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:12 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:13.799107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:13.799928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:13.810619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:13.811363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:13.812040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:13.812906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:13.813485 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:13.813918 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:13.954831 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:13.964017 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-34c995b1-9807-45de-9960-f74144b903db tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.170617 Sep 17 04:42:13.964583 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 569/1140] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:13 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:14.991836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:14.992711 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:15.004992 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:15.005832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:15.006542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:15.007663 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:15.008405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:15.009166 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:15.147885 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:15.158157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b5c98f12-faf4-4de4-94ab-124604bb8391 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1722 microversion: 2.1 time: 0.173086 Sep 17 04:42:15.158699 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 572/1141] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:14 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1722 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:16.184316 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:16.185385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:16.193224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:16.193734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:16.194193 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:16.194926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:16.195399 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:16.195850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:16.338186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:16.351060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9d021beb-4432-4848-8fda-641c5734dcf8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1699 microversion: 2.1 time: 0.173032 Sep 17 04:42:16.351932 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 570/1142] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:16 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1699 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:16.376238 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:42:16.383275 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:16.383769 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:16.384201 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:16.446592 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Going to try to start instance {{(pid=120626) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 17 04:42:16.495016 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f6407dc0-ce8d-47e6-823c-0b242bfa82d7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action" status: 202 len: 0 microversion: 2.1 time: 0.122571 Sep 17 04:42:16.495810 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 573/1143] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:42:16 2022] POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:42:16.506776 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:16.507394 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:16.520154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:16.520642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:16.521073 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:16.521798 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:16.522252 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:16.522643 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:16.689937 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:16.698761 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-94ecb9fc-4f96-441c-a40f-e22a95594d94 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1708 microversion: 2.1 time: 0.196825 Sep 17 04:42:16.699209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 571/1144] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:16 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1708 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:17.723196 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:17.723905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:17.730930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:17.731419 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:17.731856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:17.732552 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:17.733032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:17.733479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:17.874236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:17.882993 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b06e06db-7a8a-4f63-8dfe-9b9e768c0b62 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1708 microversion: 2.1 time: 0.164172 Sep 17 04:42:17.883546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 574/1145] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:17 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1708 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:42:18.909661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:42:18.910388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:42:18.917472 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:18.917961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:18.918389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:18.919090 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:42:18.919549 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:42:18.919972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:42:19.065348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:42:19.078341 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d5cacaae-091a-41cb-b391-57c368f420ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1712 microversion: 2.1 time: 0.174259 Sep 17 04:42:19.079200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 572/1146] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:42:18 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1712 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:14.145969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:14.146974 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:14.150126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=20 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:14.151774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:14.159392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:14.159863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:14.160291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:14.160977 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:14.161447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:14.161862 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:14.246979 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lazy-loading 'services' on Instance uuid 2113221b-5936-4b6e-8e99-2dc1b3ab6815 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:43:14.324279 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-74a3aac0-e09d-412f-92bd-debd717afd9b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments/15d97420-8110-4598-9f0a-e5195077f493" status: 202 len: 0 microversion: 2.1 time: 0.185136 Sep 17 04:43:14.324872 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 575/1147] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:43:14 2022] DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments/15d97420-8110-4598-9f0a-e5195077f493 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:43:15.448372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:43:15.450818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=20 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:15.451555 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:43:15.457902 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:15.458386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:15.458826 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:15.482954 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Going to try to stop instance {{(pid=120627) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 17 04:43:15.530087 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-95bee8a7-0a82-4637-806f-ed3cc23fb634 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action" status: 202 len: 0 microversion: 2.1 time: 0.085545 Sep 17 04:43:15.530690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 573/1148] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:43:15 2022] POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:43:15.540120 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:15.540826 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:15.547950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:15.548526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:15.548922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:15.549791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:15.550497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:15.550643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:15.689443 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:15.697440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7543e6d3-5bd1-4e28-978d-5e5a91e20355 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.161488 Sep 17 04:43:15.698011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 576/1149] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:15 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:16.724367 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:16.725366 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:16.734635 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:16.735149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:16.735625 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:16.736320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:16.736770 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:16.737222 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:16.873410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:16.882177 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4ca523ce-1b6c-4a88-92e6-ea39ddbc60c0 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.164896 Sep 17 04:43:16.882762 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 574/1150] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:16 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:17.910067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:17.911073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:17.921982 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:17.922882 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:17.923540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:17.924722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:17.925526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:17.926230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:18.067470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:18.075846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7e9e3c39-cfb0-4f23-8361-cf3c692acd3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.172548 Sep 17 04:43:18.076392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 577/1151] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:17 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:19.107812 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:19.108598 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:19.116200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:19.116691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:19.117121 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:19.117855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:19.118393 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:19.118736 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:19.261744 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:19.269781 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b47a5ff7-5542-4661-9ecb-f19cdcc5fbc7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.167460 Sep 17 04:43:19.270348 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 575/1152] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:19 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:20.297175 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:20.298230 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:20.308941 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:20.309797 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:20.310576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:20.311809 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:20.312260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:20.312658 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:20.444041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:20.451829 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a721602d-40f8-48d0-b44c-aed960c26b45 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.161332 Sep 17 04:43:20.452504 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 578/1153] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:20 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:21.479806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:21.480748 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:21.491663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:21.492474 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:21.493355 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:21.494496 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:21.495265 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:21.495978 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:21.631602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:21.643596 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-46233c05-48ac-4d1d-a115-e9fe6a4d9bfa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.170047 Sep 17 04:43:21.644387 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 576/1154] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:21 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:22.671296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:22.673162 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:22.680279 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:22.680782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:22.681255 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:22.681953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:22.682411 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:22.682835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:22.985491 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:22.993720 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2d5365b0-62dc-4df3-b659-f2476fc61f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.329126 Sep 17 04:43:22.994296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 579/1155] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:22 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:24.021716 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:24.022696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:24.032624 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:24.033127 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:24.033582 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:24.034274 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:24.034728 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:24.035154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:24.230959 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:24.239315 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-737fe326-428f-47c4-a6e8-89f76fa2736e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.224517 Sep 17 04:43:24.239892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 577/1156] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:24 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:25.266684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:25.267591 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:25.278847 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:25.279651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:25.280339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:25.281565 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:25.282295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:25.282705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:25.415721 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:25.423913 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a7cfbfb5-9723-47dc-bb72-08132b5cbc3b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.163456 Sep 17 04:43:25.424289 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 580/1157] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:25 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:26.455565 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:26.456669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:26.467906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:26.468670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:26.469475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:26.470612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:26.471377 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:26.472083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:26.610225 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:26.618570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-98b44631-27a3-40cc-8545-eb2ed6273b6e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.169735 Sep 17 04:43:26.619145 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 578/1158] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:26 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:27.646010 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:27.647043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:27.657908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:27.658390 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:27.658822 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:27.659519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:27.659975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:27.660401 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:27.801223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:27.809143 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0283f27a-bd01-4623-bd7c-1d4e9d33824e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.169707 Sep 17 04:43:27.809721 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 581/1159] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:27 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:28.836495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:28.837547 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:28.848198 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:28.848989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:28.849760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:28.850899 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:28.851642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:28.852339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:28.992188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:29.000140 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1ff8057f-c553-462e-bc38-e2bc4a2e53fb tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.170507 Sep 17 04:43:29.000816 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 579/1160] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:28 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:30.036792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:30.037790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:30.048660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:30.049528 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:30.050239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:30.051180 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:30.051621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:30.052043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:30.195963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:30.204361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b10510b3-f9c6-4f60-9723-7ab40a697f1d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1676 microversion: 2.1 time: 0.174228 Sep 17 04:43:30.204925 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 582/1161] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:30 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1676 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:31.232965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:31.234021 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:31.245365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:31.246186 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:31.246926 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:31.248076 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:31.248838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:31.249600 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:31.387692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:31.399607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ca0f2fe1-6063-4ac7-b9d8-f4f246c973bd tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1653 microversion: 2.1 time: 0.173137 Sep 17 04:43:31.400374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 580/1162] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:31 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1653 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:31.422117 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:43:31.428896 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:31.429394 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:31.429856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:31.451820 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Going to try to start instance {{(pid=120626) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 17 04:43:31.495636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-78930ac2-423d-4a6e-b045-4b6c165224c1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action" status: 202 len: 0 microversion: 2.1 time: 0.077219 Sep 17 04:43:31.496127 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 583/1163] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:43:31 2022] POST /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:43:31.504597 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:31.505287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:31.511492 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:31.512019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:31.512491 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:31.513335 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:31.513799 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:31.514239 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:31.680204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:31.688498 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-8e6589e3-5593-4f3b-bd60-0465f2f77698 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1662 microversion: 2.1 time: 0.187979 Sep 17 04:43:31.689069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 581/1164] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:31 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1662 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 17 04:43:32.719972 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:32.720779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:32.728322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:32.728803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:32.729284 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:32.729960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:32.730410 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:32.730835 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:32.865170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:32.873887 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a8e99068-2c13-4759-805e-d96e29e576da tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1662 microversion: 2.1 time: 0.161766 Sep 17 04:43:32.874705 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 584/1165] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:32 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1662 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:43:33.898981 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:43:33.899704 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:43:33.909182 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:33.909918 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:33.910546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:33.911533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:43:33.912194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:43:33.912809 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:43:34.048369 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:43:34.059607 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9b2013cf-ec4f-4cdf-8ee1-c023644d2a39 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1666 microversion: 2.1 time: 0.165596 Sep 17 04:43:34.060435 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 582/1166] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:43:33 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1666 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:29.280870 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:29.283221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:29.285763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:44:29.285960 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:44:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments (ip 10.222.0.60) !!! Sep 17 04:44:29.286855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=20 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:29.287702 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=67,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:29.296846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.297355 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.297793 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.298480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.298929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.299357 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.330795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a456cf72-8a8a-480e-9d61-d08c74a40810 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053916 Sep 17 04:44:29.331347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 585/1167] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:44:29 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:44:29.398219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:29.398889 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:29.401148 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:44:29.401291 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:44:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 (ip 10.222.0.60) !!! Sep 17 04:44:29.402929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 17 04:44:29.403095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:44:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 (ip 10.222.0.60) !!! Sep 17 04:44:29.404242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=20 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:29.404918 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:29.418200 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.418685 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.419119 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.419817 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.420269 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.420694 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.447963 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Going to try to terminate instance {{(pid=120627) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:44:29.530327 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f23b7751-9cdd-495e-8fef-4ea5ff12eaff tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 204 len: 0 microversion: 2.1 time: 0.136051 Sep 17 04:44:29.530917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 583/1168] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:44:29 2022] DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:44:29.539894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:29.540499 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:29.546667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.547152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.547587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.548319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:29.548773 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:29.549254 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:29.694881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:29.703344 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-7d2f38b2-5e6f-4ee7-858b-34a4a3d4faf9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1672 microversion: 2.1 time: 0.167779 Sep 17 04:44:29.703919 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 586/1169] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:29 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1672 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:30.733019 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:30.734213 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:30.744194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:30.744669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:30.745111 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:30.745815 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:30.746262 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:30.746676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:30.904568 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:30.912657 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-47b43df1-175b-4e72-9092-0561762a81b8 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1672 microversion: 2.1 time: 0.186967 Sep 17 04:44:30.913259 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 584/1170] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:30 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1672 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:31.935819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:31.936549 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:31.943855 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:31.944341 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:31.944779 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:31.945860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:31.946322 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:31.946749 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:32.077874 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:32.086217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-2504498b-1899-44b2-a6b2-9b07a3110afa tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1672 microversion: 2.1 time: 0.154364 Sep 17 04:44:32.086791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 587/1171] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:31 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1672 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:32.677223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2113221b-5936-4b6e-8e99-2dc1b3ab6815", "name": "network-vif-deleted", "tag": "4b75ce58-1ab1-41d9-9d19-7da81d3db517"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:32.687561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:32.688358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:32.688790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:32.700819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] Creating event network-vif-deleted:4b75ce58-1ab1-41d9-9d19-7da81d3db517 for instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 on hv-cinder-85799 Sep 17 04:44:32.706281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-d860010f-c998-4139-8201-32c627072678 req-2f89b3d5-2d7b-4881-9aa3-94e921407ce6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032828 Sep 17 04:44:32.706935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 585/1172] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:44:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:44:33.109787 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:33.110544 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:33.117507 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:33.118030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:33.118457 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:33.119171 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:33.119618 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:33.120033 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:33.153457 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:33.153951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:33.154384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:33.154833 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.objects.instance [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lazy-loading 'fault' on Instance uuid 2113221b-5936-4b6e-8e99-2dc1b3ab6815 {{(pid=120626) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:44:33.197881 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:33.207610 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-60a059f5-f254-4d56-a960-666817c759ec tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 200 len: 1328 microversion: 2.1 time: 0.101713 Sep 17 04:44:33.208177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 588/1173] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:33 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 1328 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:34.237420 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:34.238572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:34.249046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:34.249915 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:34.250633 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:34.251787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:34.252541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:34.253332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:34.269281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 HTTP/1.1 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:44:34.270896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:44:34.272707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '2113221b-5936-4b6e-8e99-2dc1b3ab6815'} Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:44:34.273779 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:44:34.275484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:44:34.276981 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:44:34.276981 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-5ad9a202-8b39-49de-ae69-9c815a31238f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 404 len: 111 microversion: 2.1 time: 0.043005 Sep 17 04:44:34.276981 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 586/1174] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:34 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:44:34.281225 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e8924f39-4bec-4c0f-b6df-731a2ba51fba tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '.version_select at 0x7fbc4f27dee0>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:34.307512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e8924f39-4bec-4c0f-b6df-731a2ba51fba tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1609495567" status: 202 len: 0 microversion: 2.1 time: 0.030212 Sep 17 04:44:34.308079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 589/1175] 10.222.0.60 () {58 vars in 1221 bytes} [Sat Sep 17 04:44:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1609495567 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 17 04:44:34.986406 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-aa2802ea-0d41-4f41-af17-a4252fb6aba7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: .version_select at 0x7fbc4f567ca0>, body: {"keypair": {"name": "tempest-keypair-923581295", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJuYJJg8Ylp56JF3K5D5j367NylU5/YHvQBpMXSXwUwqz8sgV/if4QGLrqMa5161BiZame2I15IYEN6jS/qf+JwF13VkdhwvfD2jGj81d37A0CLYH7nfcQfWga/aUsKuqA=="}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:35.002176 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-aa2802ea-0d41-4f41-af17-a4252fb6aba7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for project e4563767772c486f8fbb9ea131771412. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:44:35.006035 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-aa2802ea-0d41-4f41-af17-a4252fb6aba7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for user a486a4bb6b724dacbb78a9e0fe6a789e and project e4563767772c486f8fbb9ea131771412. Resources: {'key_pairs'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:44:35.034832 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-aa2802ea-0d41-4f41-af17-a4252fb6aba7 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.052404 Sep 17 04:44:35.035450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 587/1176] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:44:34 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:44:36.349704 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-97614385", "imageRef": "40659891-bb0b-407d-9b10-1486679ade98", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "276b59e3-5161-4328-9145-0306fa8cd354"}], "security_groups": [{"name": "tempest-securitygroup--1757451277"}], "key_name": "tempest-keypair-923581295", "user_data": "}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:36.464011 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.network.neutron [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] validate_networks() for [('276b59e3-5161-4328-9145-0306fa8cd354', None, None, None, None, None)] {{(pid=120626) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:44:36.932256 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Flavor limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:44:36.932788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Image limits 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:44:36.933265 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Flavor pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:44:36.933693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Image pref 0:0:0 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:44:36.934097 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.virt.hardware [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120626) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:44:36.943837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:36.944319 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:36.944746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:36.945505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:36.945954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:36.946370 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:36.963808 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for project e4563767772c486f8fbb9ea131771412. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:44:36.968017 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Getting quotas for user a486a4bb6b724dacbb78a9e0fe6a789e and project e4563767772c486f8fbb9ea131771412. Resources: {'ram', 'cores', 'instances'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:44:36.979384 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Going to run 1 instances... {{(pid=120626) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:44:36.981785 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='40659891-bb0b-407d-9b10-1486679ade98',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120626) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:44:37.037496 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR oslo.messaging._drivers.impl_rabbit [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [e251d35d-762b-4a15-a983-8b0a90ba6542] AMQP server on 10.222.0.60:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 17 04:44:38.050722 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [e251d35d-762b-4a15-a983-8b0a90ba6542] Reconnected to AMQP server on 10.222.0.60:5672 via [amqp] client with port 37336. Sep 17 04:44:38.058471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-85cd8b46-a784-4362-ad33-faed8810319f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.713404 Sep 17 04:44:38.059128 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 590/1177] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:44:36 2022] POST /compute/v2.1/servers => generated 402 bytes in 1715 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:44:38.070288 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-cc678f57-ea16-49af-9c70-25dcf35263a9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:38.070895 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-cc678f57-ea16-49af-9c70-25dcf35263a9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:38.129448 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-cc678f57-ea16-49af-9c70-25dcf35263a9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:38.136423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-cc678f57-ea16-49af-9c70-25dcf35263a9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1235 microversion: 2.1 time: 0.069992 Sep 17 04:44:38.136986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 588/1178] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:38 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1235 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:39.159441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:39.160775 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:39.167876 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:39.168439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:39.168906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:39.169893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:39.170342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:39.170792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:39.240995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:39.249989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a375d2eb-d38c-49e8-a8ce-00595ec9d62b tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1305 microversion: 2.1 time: 0.095018 Sep 17 04:44:39.250576 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 591/1179] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:39 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1305 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:40.273983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:40.274630 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:40.281856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:40.282350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:40.282852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:40.283557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:40.284144 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:40.284603 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:40.514822 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:40.522682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c9acda9a-ce48-4d60-a20f-dcb1bd70b740 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.252664 Sep 17 04:44:40.523637 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 589/1180] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:40 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:40.674439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4951bb59-de70-4ddd-8dd5-37d9faceb6f9", "tag": "a18bc2f1-2476-4adc-a624-1c0a1132e305"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:40.685794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:40.686375 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:40.686893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:40.697951 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] Creating event network-changed:a18bc2f1-2476-4adc-a624-1c0a1132e305 for instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 on hv-cinder-85799 Sep 17 04:44:40.704001 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-52643022-63e0-4af0-a72a-3003440eddec req-a61df523-7d0d-4128-a7f9-05cb63a49a1c service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039977 Sep 17 04:44:40.704794 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 592/1181] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:44:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:44:41.548083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:41.548778 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:41.557266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:41.557859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:41.558396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:41.559246 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:41.559806 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:41.560332 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:41.694859 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:41.702989 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bdeeb67c-fdd8-4ff7-aaf9-6247f3f752d5 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.160136 Sep 17 04:44:41.703528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 590/1182] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:41 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:42.732021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:42.733219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:42.743927 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:42.744733 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:42.745505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:42.746681 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:42.747132 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:42.747554 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:42.878680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:42.886481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-94bfc1b9-853c-4e52-a035-845cbca055b3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.162450 Sep 17 04:44:42.887056 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 593/1183] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:42 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:43.909242 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:43.909894 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:43.916986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:43.917506 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:43.917961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:43.918657 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:43.919184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:43.919615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:44.094868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:44.104652 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1e6d6379-e93d-4139-9d01-f4b149ad09e2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.199413 Sep 17 04:44:44.105468 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 591/1184] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:43 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:45.131995 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:45.132976 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:45.142015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:45.142539 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:45.142978 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:45.143679 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:45.144134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:45.144566 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:45.289904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:45.297844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0192ea91-5cdd-48ec-ba5c-27b4cd18191a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.172605 Sep 17 04:44:45.298416 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 594/1185] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:45 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:45.779928 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4951bb59-de70-4ddd-8dd5-37d9faceb6f9", "name": "network-vif-plugged", "status": "completed", "tag": "a18bc2f1-2476-4adc-a624-1c0a1132e305"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:45.790674 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:45.791188 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:45.791608 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:45.805156 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] Creating event network-vif-plugged:a18bc2f1-2476-4adc-a624-1c0a1132e305 for instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 on hv-cinder-85799 Sep 17 04:44:45.810436 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-0375d056-2b10-4b63-8027-8943f9022d76 req-a55a64eb-aff7-483a-b813-60b4d2ff113d service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034442 Sep 17 04:44:45.811027 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 592/1186] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:44:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:44:46.333582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:46.334943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:46.343991 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:46.344474 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:46.344911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:46.345633 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:46.346086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:46.346512 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:46.482031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:46.490342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-12204363-cd4d-4371-8639-589d3a3f0e66 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1361 microversion: 2.1 time: 0.163335 Sep 17 04:44:46.490922 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 595/1187] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:46 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1361 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:47.518556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:47.519476 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:47.527896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:47.528546 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:47.529181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:47.530263 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:47.531185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:47.531760 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:47.669554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:44:47.677619 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d1a6655b-d758-4969-bcf1-c095f605b732 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1545 microversion: 2.1 time: 0.166820 Sep 17 04:44:47.678175 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 593/1188] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:44:47 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1545 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:44:47.700884 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:44:47.701511 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:44:47.703498 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:47.704175 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=6 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:47.711086 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:47.711574 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:47.712014 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:47.712712 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:47.713199 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:47.713709 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:47.792444 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-402635d7-4fd7-41a4-a310-6bc320a45f88 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096180 Sep 17 04:44:47.793236 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 596/1189] 10.222.0.60 () {58 vars in 1275 bytes} [Sat Sep 17 04:44:47 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-interface => generated 284 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:44:49.026663 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4951bb59-de70-4ddd-8dd5-37d9faceb6f9", "tag": "a18bc2f1-2476-4adc-a624-1c0a1132e305"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:44:49.031602 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:44:49.038297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:44:49.038771 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:44:49.039194 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:44:49.051938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] Creating event network-changed:a18bc2f1-2476-4adc-a624-1c0a1132e305 for instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 on hv-cinder-85799 Sep 17 04:44:49.060109 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-ad7fce2c-6616-4884-902d-fc19aa1153da req-3ba1aa51-5291-47c9-979b-6775c9a561f1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037061 Sep 17 04:44:49.061018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 594/1190] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:44:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:45:53.735085 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=40,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:45:53.736222 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=58 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:45:53.807943 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:45:53.808434 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:45:53.814903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:45:53.815374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:45:53.815800 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:45:53.816485 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:45:53.816930 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:45:53.817386 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:45:53.841243 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-332e5f60-5e19-4209-932c-f9ba6f1414a1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/ips" status: 200 len: 144 microversion: 2.1 time: 0.109334 Sep 17 04:45:53.841798 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 597/1191] 10.222.0.60 () {58 vars in 1248 bytes} [Sat Sep 17 04:45:53 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/ips => generated 144 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:45:55.196836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:45:55.197845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:45:55.202642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4e25a77a-03e4-43fc-a27b-7a6d2eb0383a"}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:45:55.205066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:45:55.213309 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:45:55.213856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:45:55.214405 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:45:55.215134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:45:55.215589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:45:55.216063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:45:55.487524 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f03967a3-6eea-44bf-b1a8-bca0e2f7936e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.292696 Sep 17 04:45:55.488134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 595/1192] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:45:55 2022] POST /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 194 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:03.273183 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:03.275525 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:03.282429 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:03.282910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:03.283345 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:03.284030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:03.284480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:03.284908 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:03.316505 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-e69f3de8-c66e-42e0-98ee-226c8885e17e tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047306 Sep 17 04:46:03.317067 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 598/1193] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:46:03 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:03.329807 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:03.330381 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:03.338083 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:03.338610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:03.339082 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:03.339788 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:03.340346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:03.340774 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:03.371104 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-1a4a1533-79d3-4f2d-ba32-a527afb6f6ad tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a" status: 200 len: 194 microversion: 2.1 time: 0.046347 Sep 17 04:46:03.371683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 596/1194] 10.222.0.60 () {58 vars in 1413 bytes} [Sat Sep 17 04:46:03 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:04.741296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cb32a4c1-7be0-4345-a9f4-14691226bd1b"}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:46:04.743686 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:04.750167 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:04.750655 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:04.751091 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:04.751782 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:04.752234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:04.752660 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:05.050326 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-f2801d13-2d5b-4fa1-961b-2ec1570aaa44 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "POST /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.312859 Sep 17 04:46:05.050931 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 599/1195] 10.222.0.60 () {60 vars in 1323 bytes} [Sat Sep 17 04:46:04 2022] POST /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 194 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:12.650028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:12.652340 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:12.659525 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.659998 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.660434 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:12.661167 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.661647 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.662067 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:12.693123 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-09ad2839-bcf7-4347-a0d3-f08f785b130d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.047086 Sep 17 04:46:12.693787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 597/1196] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:46:12 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 371 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:46:12.849954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:12.850863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:12.860340 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.860831 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.861313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:12.862006 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.862460 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.862886 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:12.892003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3a35c0bf-bad8-4d0a-af17-dec7f652035a tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a" status: 200 len: 194 microversion: 2.1 time: 0.050803 Sep 17 04:46:12.892562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 600/1197] 10.222.0.60 () {58 vars in 1413 bytes} [Sat Sep 17 04:46:12 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:12.903189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:12.903690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:12.909972 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.910453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.910892 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:12.911589 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:12.912043 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:12.912470 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:13.005124 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lazy-loading 'services' on Instance uuid 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:46:13.088528 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-f5853d68-c961-408f-9a71-143a3e26ea24 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a" status: 202 len: 0 microversion: 2.1 time: 0.189147 Sep 17 04:46:13.089102 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 598/1198] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:46:12 2022] DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/4e25a77a-03e4-43fc-a27b-7a6d2eb0383a => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:46:15.292612 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:15.293165 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:15.300072 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:15.300547 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:15.300975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:15.301693 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:15.302116 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:15.302540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:15.333865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0d440332-70d1-45e6-8b69-20e5800b414d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/cb32a4c1-7be0-4345-a9f4-14691226bd1b" status: 200 len: 194 microversion: 2.1 time: 0.045295 Sep 17 04:46:15.334471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 601/1199] 10.222.0.60 () {58 vars in 1413 bytes} [Sat Sep 17 04:46:15 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/cb32a4c1-7be0-4345-a9f4-14691226bd1b => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:15.349718 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:15.350255 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:15.356575 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:15.357061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:15.357521 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:15.358208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:15.358661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:15.359082 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:15.441358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lazy-loading 'services' on Instance uuid 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:46:15.517099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a1640a8e-d592-4242-a8e1-232b52a087fe tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/cb32a4c1-7be0-4345-a9f4-14691226bd1b" status: 202 len: 0 microversion: 2.1 time: 0.172159 Sep 17 04:46:15.517757 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 599/1200] 10.222.0.60 () {58 vars in 1416 bytes} [Sat Sep 17 04:46:15 2022] DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments/cb32a4c1-7be0-4345-a9f4-14691226bd1b => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:46:16.695989 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:16.698328 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:16.705140 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.705632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.706075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:16.706776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.707237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.707674 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:16.739148 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-829e23a7-89f7-452f-821d-545534222be1 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047182 Sep 17 04:46:16.739717 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 602/1201] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:46:16 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:46:16.868389 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:16.870749 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:16.877473 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.877948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.878378 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:16.879060 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.879503 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.879924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:16.909292 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e6c8edd3-36c3-4913-9a8c-b4b815bb1d68 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045630 Sep 17 04:46:16.909828 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 600/1202] 10.222.0.60 () {58 vars in 1302 bytes} [Sat Sep 17 04:46:16 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 17 04:46:16.974182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:16.974680 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:16.980939 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.981488 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.981910 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:16.982601 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:16.983052 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:16.983480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:17.009124 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:46:17.091942 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-67b7ca9e-1959-4552-ac48-0f70bb0d30d3 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 204 len: 0 microversion: 2.1 time: 0.121422 Sep 17 04:46:17.092530 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 603/1203] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:46:16 2022] DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:46:17.101039 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:17.101612 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:17.107651 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:17.108129 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:17.108557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:17.109299 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:17.109765 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:17.110168 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:17.242702 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:46:17.254296 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-54c203c5-7177-4347-8e32-c28c0ef3fa0d tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1670 microversion: 2.1 time: 0.157015 Sep 17 04:46:17.255028 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 601/1204] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:17 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1670 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:46:18.287380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:18.288461 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:18.299347 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:18.300168 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:18.300893 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:18.302102 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:18.302894 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:18.303568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:18.449582 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:46:18.458235 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-0e865548-9d04-4f5a-a2b0-20de215318ea tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1670 microversion: 2.1 time: 0.177896 Sep 17 04:46:18.458814 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 604/1205] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:18 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1670 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:46:19.482834 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:19.483407 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:19.490294 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:19.490786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:19.491226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:19.491947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:19.492401 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:19.492804 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:19.723856 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:46:19.731966 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d8f1cf36-6a97-4fd1-a5d6-ca7c69a71d35 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1670 microversion: 2.1 time: 0.253126 Sep 17 04:46:19.732577 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 602/1206] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:19 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1670 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:46:20.301404 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4951bb59-de70-4ddd-8dd5-37d9faceb6f9", "name": "network-vif-deleted", "tag": "a18bc2f1-2476-4adc-a624-1c0a1132e305"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:46:20.312892 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:20.313519 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:20.314016 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:20.326114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] Creating event network-vif-deleted:a18bc2f1-2476-4adc-a624-1c0a1132e305 for instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 on hv-cinder-85799 Sep 17 04:46:20.332738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-9a4689b7-1852-40d8-b338-1f87ea232cd8 req-165b3d3f-5699-4c6c-bedc-8ded7db872ac service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035879 Sep 17 04:46:20.334513 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 605/1207] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:46:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:46:20.756495 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:20.757040 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:20.764036 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:20.764536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:20.765071 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:20.765898 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:20.766378 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:20.766795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:20.798820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:20.799543 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:20.800219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:20.800667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lazy-loading 'fault' on Instance uuid 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:46:20.842967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a486a4bb6b724dacbb78a9e0fe6a789e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e4563767772c486f8fbb9ea131771412', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:46:20.851787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-07764761-1384-4cfa-a49b-029d4fb92483 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 200 len: 1325 microversion: 2.1 time: 0.100396 Sep 17 04:46:20.852706 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 603/1208] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:20 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 1325 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:46:21.879973 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:21.881030 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:21.889188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:21.889777 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:21.890306 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:21.891217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:21.891767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:21.892288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:21.909112 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 HTTP/1.1 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:46:21.910760 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:21.912883 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '4951bb59-de70-4ddd-8dd5-37d9faceb6f9'} Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:46:21.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:21.916470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:21.918002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:46:21.918002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-ef3d7eaf-98fb-4f85-9f5c-d0e3eac3662f tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 404 len: 111 microversion: 2.1 time: 0.039348 Sep 17 04:46:21.918002 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 606/1209] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:21 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:46:21.921247 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-95a5f7e2-3e4f-4203-8903-bf3d42f04fa9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '.version_select at 0x7fbc4f4439d0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:21.943207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-95a5f7e2-3e4f-4203-8903-bf3d42f04fa9 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-923581295" status: 202 len: 0 microversion: 2.1 time: 0.025882 Sep 17 04:46:21.943734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 604/1210] 10.222.0.60 () {58 vars in 1218 bytes} [Sat Sep 17 04:46:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-923581295 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:46:24.867064 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:24.867597 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:24.870093 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:24.875667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:24.876148 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:24.876580 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:24.877303 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:24.877755 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:24.878176 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:24.881616 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:24.894423 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 HTTP/1.1 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:46:24.896075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.897863 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '4951bb59-de70-4ddd-8dd5-37d9faceb6f9'} Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:46:24.901398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.903100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.904691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:46:24.904691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-adc468f5-6eb6-4368-90a5-a3f8cf26149c tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 404 len: 111 microversion: 2.1 time: 0.034334 Sep 17 04:46:24.904691 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 607/1211] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:46:24 2022] DELETE /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:46:24.906134 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:24.906754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 4951bb59-de70-4ddd-8dd5-37d9faceb6f9] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:24.914382 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:24.914867 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:24.915308 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:24.915997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:24.916455 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:24.916884 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:24.930324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 HTTP/1.1 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:46:24.931920 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.933516 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '4951bb59-de70-4ddd-8dd5-37d9faceb6f9'} Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:46:24.934469 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. Sep 17 04:46:24.935994 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:24.937453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 4951bb59-de70-4ddd-8dd5-37d9faceb6f9 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:46:24.937453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b0930dca-9d50-4e89-8022-9d6f6507dcc2 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9" status: 404 len: 111 microversion: 2.1 time: 0.031144 Sep 17 04:46:24.937453 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 605/1212] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:24 2022] GET /compute/v2.1/servers/4951bb59-de70-4ddd-8dd5-37d9faceb6f9 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:46:26.092774 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:26.093351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:26.100069 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:26.100551 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:26.100980 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:26.101666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:26.102152 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:26.102568 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:26.116330 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 HTTP/1.1 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:46:26.117901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.119636 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '2113221b-5936-4b6e-8e99-2dc1b3ab6815'} Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:46:26.120672 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.122087 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.123746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:46:26.123746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4e53cf7e-104e-4c05-a8f2-bb2cf9e6de10 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 404 len: 111 microversion: 2.1 time: 0.030759 Sep 17 04:46:26.123746 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 608/1213] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:46:26 2022] DELETE /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:46:26.129243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:46:26.129922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] [instance: 2113221b-5936-4b6e-8e99-2dc1b3ab6815] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:46:26.132388 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=26,nova_api:DELETE=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:26.138422 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:26.138908 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:26.139350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:26.140049 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:46:26.140548 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:46:26.140985 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:46:26.144866 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:46:26.158223 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.wsgi [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] HTTP exception thrown: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Request method failure captured: Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: request: GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 HTTP/1.1 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept: application/json Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Accept-Encoding: identity Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Connection: close Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Length: 0 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Content-Type: application/json Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Host: 10.222.0.60 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Openstack-System-Scope: None Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: User-Agent: python-urllib3/1.26.12 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Auth-Token: gAAAAABjJU-gC4fpbwKq1fKwH8UKBtbJNomwJ1XXiJaq5sFtw_qpORhDVppxQ_SxEXr7Hx-CjgZiVY9DWUYctrNsgElSSksYo9IWvLGGSoEsT5Lz-bNDIKrgIKQ6eF6GnQb0YVlT84WOSJpkCfRfNv1waEQzqADODIPN-fXGftyy1j3Y4M2LDh8 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Id: None Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Domain-Name: None Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Identity-Status: Confirmed Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Is-Admin-Project: True Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Id: default Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Domain-Name: Default Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Project-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Role: reader,member Sep 17 04:46:26.160094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Roles: reader,member Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e4563767772c486f8fbb9ea131771412"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e4563767772c486f8fbb9ea131771412"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e4563767772c486f8fbb9ea131771412"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Id: e4563767772c486f8fbb9ea131771412 Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-Tenant-Name: tempest-AttachVolumeTestJSON-109305465 Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Id: default Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Domain-Name: Default Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Id: a486a4bb6b724dacbb78a9e0fe6a789e Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: X-User-Name: tempest-AttachVolumeTestJSON-109305465-project Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: method: > Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: exception: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.161864 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: action_args: {'id': '2113221b-5936-4b6e-8e99-2dc1b3ab6815'} Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:46:26.162850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. Sep 17 04:46:26.164509 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR nova.api.openstack.wsgi Sep 17 04:46:26.165975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] Returning 404 to user: Instance 2113221b-5936-4b6e-8e99-2dc1b3ab6815 could not be found. {{(pid=120627) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:46:26.165975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-72d6d4fd-034d-443c-a5e4-ae60b0bbe6c6 tempest-AttachVolumeTestJSON-109305465 tempest-AttachVolumeTestJSON-109305465-project] 10.222.0.60 "GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815" status: 404 len: 111 microversion: 2.1 time: 0.036342 Sep 17 04:46:26.165975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 606/1214] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:46:26 2022] GET /compute/v2.1/servers/2113221b-5936-4b6e-8e99-2dc1b3ab6815 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:47:28.408223 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:28.408967 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=3 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:28.636599 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c948ef02-f8e7-4633-a241-78762c11632a tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Action: 'create', calling method: .version_select at 0x7fbc4f2ca1f0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1154441948", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCvsN46afWQ55IGB+IeHZDypNx6n8lPauLghjj77C0zkLKQIXPWMLmKbWQhj/GZFCimfzsdk9AiKCtE/mIUiR68wAfpAUGy50tpkfCV6YF0JCZFX+63rAsgfv9G5NIHOvA=="}} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:47:28.654195 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-c948ef02-f8e7-4633-a241-78762c11632a tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Getting quotas for project e2e4abf23c7a495cb117b10045236a1c. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:47:28.658475 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.quota [None req-c948ef02-f8e7-4633-a241-78762c11632a tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Getting quotas for user bbdf6fa7e7f649e3a854544c7e1fe36c and project e2e4abf23c7a495cb117b10045236a1c. Resources: {'key_pairs'} {{(pid=120626) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:47:28.684778 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c948ef02-f8e7-4633-a241-78762c11632a tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.278126 Sep 17 04:47:28.685442 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 609/1215] 10.222.0.60 () {60 vars in 1159 bytes} [Sat Sep 17 04:47:28 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:47:49.240395 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:49.244991 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-370455216", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1154441948", "security_groups": [{"name": "tempest-secgroup-smoke-650613939"}], "networks": [{"uuid": "1a206bad-97c5-44c1-b0a5-c29328556d8b"}], "block_device_mapping_v2": [{"uuid": "9769d0e3-a106-440c-aeb5-46d08e252782", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:47:49.359664 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.network.neutron [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] validate_networks() for [('1a206bad-97c5-44c1-b0a5-c29328556d8b', None, None, None, None, None)] {{(pid=120627) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 17 04:47:49.837922 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Flavor limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 17 04:47:49.838569 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Image limits 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 17 04:47:49.839172 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Flavor pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 17 04:47:49.839551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Image pref 0:0:0 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 17 04:47:49.839960 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.virt.hardware [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120627) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 17 04:47:49.849713 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:49.850231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:49.850676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:49.851407 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:49.851862 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:49.852281 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:49.869940 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Getting quotas for project e2e4abf23c7a495cb117b10045236a1c. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 17 04:47:49.874189 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.quota [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Getting quotas for user bbdf6fa7e7f649e3a854544c7e1fe36c and project e2e4abf23c7a495cb117b10045236a1c. Resources: {'ram', 'cores', 'instances'} {{(pid=120627) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 17 04:47:49.932196 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Going to run 1 instances... {{(pid=120627) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 17 04:47:49.934554 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9769d0e3-a106-440c-aeb5-46d08e252782',volume_size=None,volume_type=None)] {{(pid=120627) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 17 04:47:50.069150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR oslo.messaging._drivers.impl_rabbit [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [6d8fc9d6-e47e-4604-8975-d1e258edc363] AMQP server on 10.222.0.60:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 17 04:47:51.082924 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [6d8fc9d6-e47e-4604-8975-d1e258edc363] Reconnected to AMQP server on 10.222.0.60:5672 via [amqp] client with port 38362. Sep 17 04:47:51.091219 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-a68fff40-ba2d-418c-8525-7e7567928ffd tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.852562 Sep 17 04:47:51.092217 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 607/1216] 10.222.0.60 () {60 vars in 1147 bytes} [Sat Sep 17 04:47:49 2022] POST /compute/v2.1/servers => generated 405 bytes in 1854 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 17 04:47:51.106269 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:47:51.106815 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:47:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 (ip 10.222.0.60) !!! Sep 17 04:47:51.107417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=4 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:51.108348 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:47:51.112617 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-09be2b56-7096-44f7-a113-5e621f0111b7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:51.113260 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-09be2b56-7096-44f7-a113-5e621f0111b7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:51.174197 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-09be2b56-7096-44f7-a113-5e621f0111b7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:51.181609 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-09be2b56-7096-44f7-a113-5e621f0111b7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1088 microversion: 2.1 time: 0.077496 Sep 17 04:47:51.182157 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 610/1217] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:51 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1088 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:52.204746 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:52.205331 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:52.211968 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:52.212450 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:52.212875 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:52.213656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:52.214209 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:52.214627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:52.316032 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:52.325224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-375de9e9-a3f6-4cd1-9802-0c656f81c288 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1194 microversion: 2.1 time: 0.125007 Sep 17 04:47:52.325906 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 608/1218] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:52 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1194 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:53.349795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:53.350377 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:53.358804 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:53.359301 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:53.359738 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:53.360436 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:53.360888 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:53.361338 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:53.497902 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:53.506569 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-4f0bf47d-cb9d-4a26-96ad-2315fa20a039 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.162303 Sep 17 04:47:53.507169 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 611/1219] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:53 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:53.790715 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:47:53.800414 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:53.800896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:53.801346 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:53.812456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] Creating event network-changed:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:47:53.817610 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-34def649-a513-4fda-818f-6b2b77c9fe8b req-6c118e6f-6550-4516-a218-b807b2d9e5a6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093346 Sep 17 04:47:53.818204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 609/1220] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:47:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:47:54.534134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:54.535380 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:54.546312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:54.547114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:54.547828 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:54.548975 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:54.549768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:54.550479 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:54.698220 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:54.706619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bde8e2f7-744a-409c-b191-c18ab5b4fe51 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.178883 Sep 17 04:47:54.707190 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 612/1221] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:54 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:55.734907 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:55.736056 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:55.746836 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:55.747627 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:55.748103 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:55.748830 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:55.749329 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:55.749751 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:55.896061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:55.904661 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3c59e63d-8484-48ac-a954-f1350d222c35 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.177627 Sep 17 04:47:55.905266 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 610/1222] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:55 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:56.929288 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:56.929901 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:56.938981 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:56.939659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:56.940246 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:56.941405 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:56.941964 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:56.942425 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:57.080818 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:57.089244 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a53980d7-c0a2-4759-a9f8-3124dcecbe4e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.165752 Sep 17 04:47:57.089825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 613/1223] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:56 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:58.115719 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:58.116785 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:58.127676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:58.128480 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:58.129297 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:58.130396 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:58.131150 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:58.131852 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:58.273268 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:58.281669 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-3e05bc00-35a0-48d8-8648-d80f49e36b8c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.172590 Sep 17 04:47:58.282233 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 611/1224] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:58 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:47:59.312154 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:47:59.313161 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:47:59.323965 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:59.324761 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:59.325548 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:59.326666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:47:59.327433 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:47:59.328141 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:47:59.466398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:47:59.474484 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-39f99b77-ef4c-4a9a-bc25-438b6a342418 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.169123 Sep 17 04:47:59.475055 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 614/1225] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:47:59 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:00.500992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:00.502095 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:00.505900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=2 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:00.506992 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell0:SELECT=6 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:00.516138 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:00.516624 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:00.517066 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:00.517790 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:00.518243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:00.518667 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:00.653698 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:00.661967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-249a8696-2a8d-4a02-9185-a1461a677a8f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.167591 Sep 17 04:48:00.662618 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 612/1226] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:00 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:01.684217 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:01.684846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:01.687281 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell0:SELECT=2 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:48:01.692844 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:01.693351 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:01.693790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:01.694471 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:01.694933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:01.695356 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:01.835803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:01.847767 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b3e53487-2893-4a86-ab03-a76574e3cc82 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.167522 Sep 17 04:48:01.848349 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 615/1227] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:01 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:02.877018 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:02.877668 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:02.884537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:02.885029 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:02.885490 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:02.886216 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:02.886676 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:02.887101 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:03.026169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:03.034358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b48dba9a-e82e-4615-be11-ad856672930f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.162721 Sep 17 04:48:03.034929 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 613/1228] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:02 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:04.060997 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:04.062048 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:04.074716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:04.074716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:04.075135 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:04.076257 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:04.077036 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:04.077929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:04.219936 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:04.227683 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-483cdc76-6daf-4f46-a38e-52b2448a19fc tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.173104 Sep 17 04:48:04.228253 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 616/1229] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:04 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:05.250820 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:05.251519 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:05.258419 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:05.258891 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:05.259350 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:05.260052 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:05.260511 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:05.260938 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:05.396357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:05.404909 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b70fe5d8-f430-4bd0-aebb-ecabd29e967f tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.157510 Sep 17 04:48:05.405517 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 614/1230] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:05 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:06.431481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:06.432422 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:06.443480 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:06.444271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:06.445008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:06.446200 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:06.446968 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:06.447682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:06.809967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "name": "network-vif-plugged", "status": "completed", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:48:06.820505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:06.820808 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:06.821276 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:06.832975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] Creating event network-vif-plugged:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:48:06.838086 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-d2a31ec0-23cd-4da1-b337-f6dc2a8dc563 req-b715a63d-900a-4bb7-9dc0-e2509c14e3c5 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096247 Sep 17 04:48:06.838682 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 615/1231] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:48:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:48:06.866079 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:06.877852 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-28608fad-2aff-4e7e-b2ca-690fe8db8283 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.452897 Sep 17 04:48:06.878603 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 617/1232] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:06 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:07.909439 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:07.910580 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:07.921118 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:07.921935 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:07.922656 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:07.923570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:07.924020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:07.924511 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:08.062578 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:08.070592 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-4035c885-4ee3-4985-b018-352abf981fc7 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1259 microversion: 2.1 time: 0.169070 Sep 17 04:48:08.071158 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 616/1233] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:07 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1259 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:09.097134 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:09.097949 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:09.106763 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:09.107440 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:09.108015 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:09.108921 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:09.109560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:09.110125 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:09.253050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:09.261447 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a2e580a8-c123-4f45-8670-a78484cc7779 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1442 microversion: 2.1 time: 0.170413 Sep 17 04:48:09.262016 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 618/1234] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:09 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1442 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:09.283191 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:48:09.284846 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:48:09.294623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:09.295125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:09.295532 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:09.296244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:09.296691 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:09.297115 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:09.432458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:48:09.451905 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-d389519c-02f7-4a60-9156-25a66748f565 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1442 microversion: 2.1 time: 0.172233 Sep 17 04:48:09.452642 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 617/1235] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:48:09 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1442 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:48:11.214697 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:48:11.224361 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:48:11.224842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:48:11.225312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:48:11.236640 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] Creating event network-changed:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:48:11.241928 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-60917d5a-69ae-4ae4-a628-f58678864627 req-1ec58075-1dad-4dfa-9901-aef186a33bed service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094646 Sep 17 04:48:11.242661 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 619/1236] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:48:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:50:25.085961 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: ERROR oslo.messaging._drivers.impl_rabbit [-] [fcccc26a-6b9f-499f-a3bb-c4e2115ec3f4] AMQP server on 10.222.0.60:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 17 04:50:25.086794 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:25.087551 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=55 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:25.088721 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:50:25.088897 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: Sat Sep 17 04:50:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8/action (ip 10.222.0.60) !!! Sep 17 04:50:25.090850 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Sep 17 04:50:25.098385 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:50:25.099044 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:25.113675 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:25.114282 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:25.114729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:25.115437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:25.115893 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:25.116320 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:25.141855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lazy-loading 'flavor' on Instance uuid 38110d84-13ff-4555-accc-a20ccf6f1bc8 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:50:25.218983 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-c3dfd125-926d-4eda-95f8-d481ca6323ee tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "POST /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8/action" status: 202 len: 0 microversion: 2.1 time: 0.134876 Sep 17 04:50:25.219570 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 618/1237] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:50:25 2022] POST /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8/action => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 17 04:50:25.226953 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR oslo.messaging._drivers.impl_rabbit [-] [25ca7697-7b80-46d6-aaf3-fa3b6d967bbf] AMQP server on 10.222.0.60:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Sep 17 04:50:25.227516 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=22 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:25.228129 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=50 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:25.229706 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 17 04:50:25.229845 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Sat Sep 17 04:50:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 (ip 10.222.0.60) !!! Sep 17 04:50:25.237792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:25.238538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:25.246587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:25.247065 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:25.247497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:25.248188 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:25.248639 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:25.249061 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:25.447776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:25.456008 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fed16063-861c-42e9-80e1-978690bd2b2d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.231696 Sep 17 04:50:25.456573 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 620/1238] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:25 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:26.483951 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:26.484650 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:26.492708 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:26.493206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:26.493631 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:26.494312 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:26.494755 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:26.495166 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:26.500571 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO oslo.messaging._drivers.impl_rabbit [-] [fcccc26a-6b9f-499f-a3bb-c4e2115ec3f4] Reconnected to AMQP server on 10.222.0.60:5672 via [amqp] client with port 38490. Sep 17 04:50:26.682533 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:26.690437 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-49b3d665-a092-47a3-950f-7969e1b96e61 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.213692 Sep 17 04:50:26.690975 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 619/1239] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:26 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:27.714109 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:27.714731 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:27.721963 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:27.722439 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:27.722873 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:27.723665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:27.724114 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:27.724538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:27.741118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [-] [25ca7697-7b80-46d6-aaf3-fa3b6d967bbf] Reconnected to AMQP server on 10.222.0.60:5672 via [amqp] client with port 38496. Sep 17 04:50:27.861666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:27.870013 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-37cc9b38-47d2-4907-8ee0-723639294f71 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.159936 Sep 17 04:50:27.870628 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 621/1240] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:27 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:28.903226 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:28.903865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:28.912208 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:28.912707 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:28.913143 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:28.913860 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:28.914313 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:28.914734 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:29.063747 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:29.072203 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-2c7dd696-93a7-42c7-901a-238d03b4ad55 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.173499 Sep 17 04:50:29.072772 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 620/1241] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:28 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:30.100206 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:30.101261 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:30.112237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:30.113031 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:30.113860 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:30.114727 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:30.115497 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:30.115935 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:30.268532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:30.280019 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-c2d4ce63-7212-47c1-a4a2-e6ecb0c8b42e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.186126 Sep 17 04:50:30.280832 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 622/1242] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:30 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:31.315022 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:31.315997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:31.323946 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:31.324440 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:31.324873 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:31.325609 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:31.326061 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:31.326484 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:31.467572 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:31.475796 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-59b89123-0e72-4b52-9f01-22f3ae993dde tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.167535 Sep 17 04:50:31.476357 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 621/1243] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:31 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:32.503214 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:32.504123 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:32.514911 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:32.515715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:32.516445 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:32.517659 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:32.518415 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:32.519126 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:32.662438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:32.671237 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-dacf1732-2b80-4b2b-8a04-ec3ae2aca11b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.174770 Sep 17 04:50:32.671823 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 623/1244] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:32 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:33.694264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:33.694953 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:33.701931 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:33.702425 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:33.702868 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:33.703930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:33.704374 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:33.704801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:33.837692 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:33.849566 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-0e56bfbd-7a78-47dc-9bb2-06b2f27696ad tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.159128 Sep 17 04:50:33.850307 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 622/1245] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:33 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:34.889050 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:34.890073 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:34.898792 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:34.899273 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:34.899703 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:34.900391 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:34.900824 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:34.901273 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:35.042203 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:35.050650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-b44b8aa1-374b-4313-ae06-ccf77f941eeb tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.168111 Sep 17 04:50:35.051239 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 624/1246] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:34 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:36.075011 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:36.075670 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:36.085475 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:36.085965 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:36.086408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:36.087091 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:36.087564 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:36.087997 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:36.224423 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:36.236697 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bd4f086c-345b-4127-aa9d-48b5080c1f18 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.165910 Sep 17 04:50:36.237628 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 623/1247] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:36 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:37.264540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:37.265540 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:37.276665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:37.277542 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:37.278271 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:37.279407 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:37.280160 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:37.280866 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:37.428177 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:37.439858 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-581d13b7-b0a0-4f64-8f76-abf5dbbbfc06 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.181489 Sep 17 04:50:37.440651 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 625/1248] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:37 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:38.468418 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:38.469386 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:38.480488 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:38.481358 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:38.482092 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:38.483244 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:38.484020 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:38.484733 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:38.620515 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:38.630069 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-70d8d9df-8a7b-43a8-8ab3-5e0eff6d451c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.168366 Sep 17 04:50:38.630638 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 624/1249] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:38 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:39.656427 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:39.657432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:39.665906 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:39.666398 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:39.666837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:39.667546 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:39.668003 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:39.668432 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:39.803468 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:39.812027 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8d2c9419-7bef-4454-94c7-28c740a73f8b tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.162148 Sep 17 04:50:39.812630 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 626/1250] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:39 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:40.837561 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:40.838153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:40.846384 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:40.846855 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:40.847323 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:40.848024 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:40.848489 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:40.848948 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:41.062110 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:41.070372 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-83768c49-5033-4a55-b6bc-2555b1469949 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.238703 Sep 17 04:50:41.070950 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 625/1251] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:40 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:42.097670 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:42.098667 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:42.109656 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:42.110470 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:42.111170 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:42.112307 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:42.113042 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:42.113856 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:42.253666 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:42.265632 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-fb91a9f6-dd31-4815-8ea6-4eaa4a7c0fce tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.174309 Sep 17 04:50:42.266364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 627/1252] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:42 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:43.290181 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:43.290818 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:43.297477 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.297967 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.298409 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.299099 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.299556 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.299979 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.433047 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:43.440837 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-071f92ef-a761-4c6e-a788-3631d19697b1 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1567 microversion: 2.1 time: 0.154692 Sep 17 04:50:43.441456 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 626/1253] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:43 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:43.463482 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:50:43.464032 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:43.470678 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.471156 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.471587 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.472267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.472716 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.473295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.569933 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120626) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 17 04:50:43.637740 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8f313395-d4aa-4529-8147-cba7f6468b16 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "POST /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8/action" status: 202 len: 0 microversion: 2.1 time: 0.178037 Sep 17 04:50:43.638684 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 628/1254] 10.222.0.60 () {60 vars in 1278 bytes} [Sat Sep 17 04:50:43 2022] POST /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8/action => generated 0 bytes in 180 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 17 04:50:43.651154 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:43.651801 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:43.658339 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.659338 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.659471 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.660256 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:43.660754 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:43.661195 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:43.835253 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:43.843654 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9f99cb8f-1576-4116-a55e-38c880de20cf tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1515 microversion: 2.1 time: 0.197497 Sep 17 04:50:43.844231 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 627/1255] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:43 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1515 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:44.867836 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:44.868532 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:44.875075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:44.875665 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:44.876312 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:44.877379 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:44.878070 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:44.878560 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:45.024339 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:45.032648 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1c35d698-9c24-4db0-a717-d0f9a54d2e14 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1517 microversion: 2.1 time: 0.169755 Sep 17 04:50:45.033264 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 629/1256] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:44 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1517 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:45.509236 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "name": "network-vif-unplugged", "status": "completed", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:50:45.520623 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:45.521159 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:45.521896 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:45.536530 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] Creating event network-vif-unplugged:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:50:45.542152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-2fab0b32-ff92-4a2a-886f-6a2aa2a6f7b0 req-1066de28-c2b5-42d6-9f18-519d07e1f4e6 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036802 Sep 17 04:50:45.543324 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 628/1257] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:50:45 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 17 04:50:46.056219 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:46.056803 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:46.063626 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:46.064103 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:46.064533 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:46.065234 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:46.065682 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:46.066100 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:46.198851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:46.207193 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-1eee7042-a894-4f31-addf-f17a82b46c7e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.154913 Sep 17 04:50:46.207819 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 630/1258] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:46 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:47.238059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:47.239169 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:47.249840 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:47.250641 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:47.251363 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:47.252502 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:47.253301 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:47.254013 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:47.394795 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:47.403286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-14bd5afd-e552-4338-a4d8-6e6538be4017 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.173386 Sep 17 04:50:47.403858 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 629/1259] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:47 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:48.440138 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:48.441053 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:48.452311 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:48.453481 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:48.454266 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:48.455406 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:48.456137 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:48.456850 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:48.592938 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:48.600954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-00cd7cc9-83fd-4ab2-a492-2eb10ad73c2c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.167614 Sep 17 04:50:48.601562 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 631/1260] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:48 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:49.629059 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:49.630077 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:49.641152 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:49.642207 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:49.642890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:49.644228 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:49.645000 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:49.645934 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:49.780871 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:49.791531 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-475f104b-405a-41ce-b86b-88d5c08cdc67 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.168908 Sep 17 04:50:49.792218 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 630/1261] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:49 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:50.819645 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:50.820296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:50.827467 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:50.827950 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:50.828754 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:50.829472 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:50.829920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:50.830342 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:50.966163 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:50.974441 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-9c0a1d1d-04cd-46ee-ac64-cc56008e4331 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.161681 Sep 17 04:50:50.975248 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 632/1262] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:50 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:52.000833 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:52.001786 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:52.012541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:52.013404 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:52.014149 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:52.015243 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:52.015986 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:52.016684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:52.151683 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:52.159526 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-b7302a22-0ef3-4cee-96cf-2db6d42d8d6e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.165405 Sep 17 04:50:52.160088 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 631/1263] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:51 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:53.186317 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:53.187438 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:53.197302 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:53.197788 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:53.198218 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:53.198918 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:53.199374 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:53.199795 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:53.338621 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:53.346462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-3b992274-c459-426c-8928-d2c519149aab tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.166698 Sep 17 04:50:53.347043 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 633/1264] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:53 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:54.373947 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:54.374958 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:54.384684 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:54.385204 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:54.385645 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:54.386333 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:54.386787 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:54.387206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:54.523686 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:54.535365 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-bdaae964-099a-4d5d-83f4-06a305dc5a0d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.167783 Sep 17 04:50:54.536080 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 632/1265] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:54 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:55.559846 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:55.560452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:55.562859 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:55.568210 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:55.568694 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:55.569145 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:55.569837 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:55.570296 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:55.570719 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:55.574186 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:55.708701 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:55.716570 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-8c4433ea-086a-4966-9792-00d2a8a87e81 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.160961 Sep 17 04:50:55.717155 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 634/1266] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:55 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:56.744657 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:56.750192 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:56.754212 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=31 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:56.761403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:56.761977 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:56.762433 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:56.763264 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:56.763653 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:56.764094 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:56.767185 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:50:56.905536 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:56.913591 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-320ec568-aa7e-41a9-8a64-8a9e455fdb85 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.175563 Sep 17 04:50:56.914126 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 633/1267] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:56 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:57.628150 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "name": "network-vif-plugged", "status": "completed", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:50:57.638613 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:57.639118 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:57.639550 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:57.651865 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] Creating event network-vif-plugged:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:50:57.655376 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR oslo.messaging._drivers.impl_rabbit [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] [975096fe-4932-452a-89d1-e8180243976f] AMQP server on 10.222.0.60:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 17 04:50:57.940260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:57.941273 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:57.952537 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:57.953797 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:57.954541 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:57.955700 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:57.956458 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:57.957206 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:58.095917 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:58.107845 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ac14c910-6d4e-4c23-9509-c92ba0e8a734 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.173859 Sep 17 04:50:58.108522 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 634/1268] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:57 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:50:58.666462 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO oslo.messaging._drivers.impl_rabbit [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] [975096fe-4932-452a-89d1-e8180243976f] Reconnected to AMQP server on 10.222.0.60:5672 via [amqp] client with port 38572. Sep 17 04:50:58.673790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-8b46be9e-a987-49bd-ae2d-090f578e9155 req-8640a4e7-665c-4072-a339-a1c41285e6c1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 1.049199 Sep 17 04:50:58.675929 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 635/1269] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:50:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 1051 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:50:59.134342 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:50:59.135089 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:50:59.142114 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:59.142605 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:59.143107 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:59.143804 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:50:59.144260 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:50:59.144690 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:50:59.290714 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:50:59.303205 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e110807e-1cdc-4b4b-8457-d44267865147 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.174560 Sep 17 04:50:59.304113 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 635/1270] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:50:59 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:51:00.333278 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:51:00.334221 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:51:00.341825 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:51:00.342309 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:51:00.342768 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:51:00.343452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:51:00.343903 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:51:00.344327 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:51:00.483182 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:51:00.495291 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-bd60d5b0-4317-4947-b732-9cdccd189632 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1573 microversion: 2.1 time: 0.170183 Sep 17 04:51:00.496021 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 636/1271] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:51:00 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1573 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:51:01.527070 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:51:01.528055 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:51:01.539063 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:51:01.539890 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:51:01.540615 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:51:01.541819 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:51:01.542576 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:51:01.543379 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:51:01.692360 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:51:01.700566 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-547febdb-5bf0-4c86-b319-8b7c0eed1e72 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1560 microversion: 2.1 time: 0.180227 Sep 17 04:51:01.701171 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 636/1272] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:51:01 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:52:29.556353 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_api:SELECT=5 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:29.557364 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG dbcounter [-] [120626] Writing DB stats nova_cell1:SELECT=13 {{(pid=120626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:29.558459 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_api:SELECT=7 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:29.559380 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG dbcounter [-] [120627] Writing DB stats nova_cell1:SELECT=23 {{(pid=120627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 04:52:29.564041 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:29.564643 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:29.565410 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:52:29.573267 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:29.573851 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:29.574619 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:29.575526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:29.576060 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:29.576791 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:29.578888 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:29.579731 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:29.580467 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:29.594930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.compute.server_external_events [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] Creating event network-changed:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:52:29.600432 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [req-b2fe9c70-a91f-43c3-9c9e-8865204f19fb req-03cbd9fb-c2d1-406c-ae8c-cef32837eba1 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043868 Sep 17 04:52:29.601408 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 637/1273] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:52:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:52:29.608113 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Going to try to terminate instance {{(pid=120626) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 17 04:52:29.698905 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-a3fa92d5-e1e6-4269-98dd-3f8bd485ac1c tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "DELETE /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 204 len: 0 microversion: 2.1 time: 0.144345 Sep 17 04:52:29.699920 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 637/1274] 10.222.0.60 () {58 vars in 1239 bytes} [Sat Sep 17 04:52:29 2022] DELETE /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 17 04:52:29.710696 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:29.711679 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:29.719173 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:29.719911 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:29.720544 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:29.721284 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:29.721910 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:29.722505 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:29.885306 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:52:29.893153 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-ba62d78c-2120-4460-b0ce-c627df21872d tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.187881 Sep 17 04:52:29.893729 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 638/1275] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:52:29 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:52:30.916790 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:30.917501 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:30.924397 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:30.924880 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:30.925446 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:30.926075 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:30.926526 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:30.926945 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:31.068352 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.policy [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120626) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:52:31.076842 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-03cf7cff-63d0-4ccb-96eb-478f341f6118 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1566 microversion: 2.1 time: 0.164100 Sep 17 04:52:31.077452 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 638/1276] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:52:30 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1566 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:52:32.100224 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:32.100879 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:32.108046 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:32.108545 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:32.109184 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:32.109925 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:32.110442 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:32.110865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:32.325141 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:52:32.333508 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-9430a3b1-6e7b-4026-98a8-a565933ecbe5 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1448 microversion: 2.1 time: 0.237270 Sep 17 04:52:32.334096 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 639/1277] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:52:32 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1448 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:52:32.460954 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38110d84-13ff-4555-accc-a20ccf6f1bc8", "name": "network-vif-deleted", "tag": "506760d5-de67-41d5-a14c-c3b79cd54e9a"}]} {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 17 04:52:32.475142 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:32.475538 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:32.475969 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:32.488417 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.compute.server_external_events [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] Creating event network-vif-deleted:506760d5-de67-41d5-a14c-c3b79cd54e9a for instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 on hv-cinder-85799 Sep 17 04:52:32.494012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [req-f5ec3161-115b-44d2-ba64-8f510d0d15b5 req-474f7186-ba92-4da5-b80b-e2e238a295d2 service nova] 10.222.0.60 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037436 Sep 17 04:52:32.494650 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 639/1278] 10.222.0.60 () {64 vars in 1317 bytes} [Sat Sep 17 04:52:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 17 04:52:33.356477 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:33.357178 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.compute.api [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120627) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:33.363562 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:33.364125 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:33.364557 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:33.365279 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:33.365727 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:33.366590 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:33.410900 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:33.411403 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:33.411838 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG oslo_concurrency.lockutils [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120627) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:33.412286 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.objects.instance [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lazy-loading 'fault' on Instance uuid 38110d84-13ff-4555-accc-a20ccf6f1bc8 {{(pid=120627) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 17 04:52:33.456791 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.policy [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbdf6fa7e7f649e3a854544c7e1fe36c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e4abf23c7a495cb117b10045236a1c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120627) authorize /opt/stack/nova/nova/policy.py:199}} Sep 17 04:52:33.465287 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-24c601d7-971b-4270-a386-8997f3e9358e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 200 len: 1178 microversion: 2.1 time: 0.113283 Sep 17 04:52:33.465865 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 640/1279] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:52:33 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 1178 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 17 04:52:34.494728 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '>' {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:34.496012 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.compute.api [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] [instance: 38110d84-13ff-4555-accc-a20ccf6f1bc8] Fetching instance by UUID {{(pid=120626) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 17 04:52:34.506990 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:34.507776 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:34.508494 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:34.509690 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Acquiring lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 17 04:52:34.510449 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 17 04:52:34.511231 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG oslo_concurrency.lockutils [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Lock "fa9fccbc-46c5-4ec3-934d-edb50649c947" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120626) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 17 04:52:34.525295 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.wsgi [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] HTTP exception thrown: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Request method failure captured: Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: request: GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 HTTP/1.1 Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept: application/json Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Accept-Encoding: identity Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Connection: close Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Length: 0 Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Content-Type: application/json Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Host: 10.222.0.60 Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: Openstack-System-Scope: None Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: User-Agent: python-urllib3/1.26.12 Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Auth-Token: gAAAAABjJVFbapa3J9QIcZBXKbETiVH_vNzZ6H4pWnXIfxyJKcKZpzlcOqXk2jB7xJxkm8Y3wpvDU06wwT1TAWPdICfQlchfgjqbUQW2aef2AgGaODyNuSooqNxfssHmrokJwfgdcbOz2gP-zX1yg_6uLid7SPFK4S1lJhCQnGll8My4r5rcHhU Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Id: None Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Domain-Name: None Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Identity-Status: Confirmed Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Is-Admin-Project: True Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Id: default Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Domain-Name: Default Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Id: e2e4abf23c7a495cb117b10045236a1c Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Project-Name: tempest-TestShelveInstance-1751094533 Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Role: reader,member Sep 17 04:52:34.526904 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Roles: reader,member Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e2e4abf23c7a495cb117b10045236a1c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2/e2e4abf23c7a495cb117b10045236a1c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.60:8080", "publicURL": "http://10.222.0.60:8080/v1/AUTH_e2e4abf23c7a495cb117b10045236a1c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/volume/v3/e2e4abf23c7a495cb117b10045236a1c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.60/compute/v2.1"}]}] Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant: tempest-TestShelveInstance-1751094533 Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Id: e2e4abf23c7a495cb117b10045236a1c Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-Tenant-Name: tempest-TestShelveInstance-1751094533 Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User: tempest-TestShelveInstance-1751094533-project Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Id: default Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Domain-Name: Default Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Id: bbdf6fa7e7f649e3a854544c7e1fe36c Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: X-User-Name: tempest-TestShelveInstance-1751094533-project Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: method: > Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: exception: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. Sep 17 04:52:34.528730 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: action_args: {'id': '38110d84-13ff-4555-accc-a20ccf6f1bc8'} Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: {{(pid=120626) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 17 04:52:34.529987 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. Sep 17 04:52:34.531715 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: ERROR nova.api.openstack.wsgi Sep 17 04:52:34.533392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: DEBUG nova.api.openstack.wsgi [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Returning 404 to user: Instance 38110d84-13ff-4555-accc-a20ccf6f1bc8 could not be found. {{(pid=120626) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 17 04:52:34.533392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: INFO nova.api.openstack.requestlog [None req-5b39de6e-c461-48fe-a635-156ae574d5f4 tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8" status: 404 len: 111 microversion: 2.1 time: 0.042146 Sep 17 04:52:34.533392 dv-cinder-857993-2-iscsi devstack@n-api.service[120626]: [pid: 120626|app: 0|req: 640/1280] 10.222.0.60 () {58 vars in 1236 bytes} [Sat Sep 17 04:52:34 2022] GET /compute/v2.1/servers/38110d84-13ff-4555-accc-a20ccf6f1bc8 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 17 04:52:34.694120 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: DEBUG nova.api.openstack.wsgi [None req-e4972c36-0187-4757-a3ac-b128760c400e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] Calling method '.version_select at 0x7fbc4f41a9d0>' {{(pid=120627) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 17 04:52:34.719930 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: INFO nova.api.openstack.requestlog [None req-e4972c36-0187-4757-a3ac-b128760c400e tempest-TestShelveInstance-1751094533 tempest-TestShelveInstance-1751094533-project] 10.222.0.60 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1154441948" status: 202 len: 0 microversion: 2.1 time: 0.029565 Sep 17 04:52:34.720483 dv-cinder-857993-2-iscsi devstack@n-api.service[120627]: [pid: 120627|app: 0|req: 641/1281] 10.222.0.60 () {58 vars in 1254 bytes} [Sat Sep 17 04:52:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1154441948 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)